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The  Instructional  Technology  Systems  Technical  Area  of  the  U.S.  Army 
Research  Institute  For  the  Behavioral  and  Social  Sciences,  in  coordination 
with  the  Adjutant  General's  Office,  sought  to  improve  the  Army's  Basic 
Skills  Education  Programs  through  the  dissemination  of  information  about 
educational  resources  to  military  educators.  This  goal  was  met  through  the 
establishment  of  the  Military  Educators  Resource  NETWORK.  The  NETWORK  was 
pilot  tested  during  an  initial  seventeen  month  operational  period,  from 
March  1983  through  July  1984.  Throughout  the  pilot  test  phase,  evaluative 
data  were  solicited  about  the  NETWORK.  This  report  describes  the  results 
of  these  activities  and  provides  a  set  of  recommendations  to  guide 
decisions  concerning  the  future  operation  of  the  NETWORK. 
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EXECUTIVE  SUMMARY 


Basic  Skills  Resource  Center:  Final  Report  of  the  Military 
Educators  Resource  NETWORK 

The  Basic  Skills  Resource  Center  (BSRC)  was  developed  and  operated  by 
InterAnier ica  Research  Associates,  Inc.  under  contract  with  the  U.S.  Army 
Research  Institute.  The  BSRC  project  has  two  interfacing  components:  the 
implementation  and  monitoring  of  applied  research  in  the  area  of  adult 
basic  skills  and  continuing  education,  and  the  design,  implementation,  and 
operation  of  an  information  service.  Following  the  completion  of  a  needs 
assessment,  a  design  plan  was  identified  for  the  operation  of  an 
information  service  entitled  "The  Military  Educators  Resource  NETWORK." 
The  design  plan  was  implemented  and  pilot  tested  for  an  initial  seventeen 
month  period.  Throughout  this  pilot  test  period  evaluative  data  were 
collected  relative  to  the  operation  of  the  NETWORK.  This  report  serves  as 
the  final  report  of  the  BSRC  information  component  and  outlines  the 
evaluation  results  on  the  initial  operation  of  the  NETWORK. 

I 

The  primary  purpose  of  this  report  is  to  provide  a  synthesis  of  the 
evaluative  data  maintained  by  the  NETWORK  and  to  identify  a  set  of 
recommendations  for  the  future  operation  of  the  NETWORK.  Throughout  the 
seventeen  month  operational  period,  project  staff  identified  a  variety  of 
internal  and  external  evaluative  data  on  the  NETWORK'S  operational 
procedures,  perceived  quality  and  use  of  services  provided  by  the  NETWORK, 
as  well  as  the  perceived  quality  of  informational  publications  developed  by 
the  NETWORK  staff.  In  total,  nine  evaluation  activities  were  conducted 


throughout  the  pilot  test  phase  related  to  the  NETWORK'S  Inquiry  Response 
and  Referral,  Publication  Development  and  Dissemination,  and  Current 
Awareness  Services  as  well  as  the  development  of  the  Computerized  Database. 
Each  evaluation  activity  is  described  in  terms  of  purpose,  focus  and 
distribution  procedures.  Results  of  the  evaluation  activities  are 
summarized  relative  to  the  NETWORK'S  clients,  user  information  needs, 
clients'  perception  of  the  NETWORK'S  services  and  use  of  the  information 
provided  by  the  NETWORK  in  response  to  their  inquiries. 

The  primary  users  or  clients  of  the  NETWORK'S  services  were  educational 
practitioners  directly  associated  with  the  education  programs  offered  by 
the  military  services.  This  group  of  ESOs,  ESSs,  and  Counselors  had  been 
identified  as  the  target  client  group  through  the  completion  of  the  needs 
assessment  activity.  Administrators  associated  with  these  educational 
programs  (e.g.,  command  and  headquarters  staff)  were  found  to  be  the  second 
most  frequent  group  of  clients  contacting  the  NETWORK.  Overall,  personnel 
affiliated  with  the  Department  of  the  Army  accounted  for  the  majority  of 
inquiries  recorded  by  the  NETWORK  staff. 

These  user  groups  preferred  to  contact  the  NETWORK  with  their  information 
requests  via  mail  and  telephone.  Although  the  majority  of  inquiries  were 
submitted  by  mail  rather  than  by  telephone,  no  set  pattern  of  preference 
was  noted.  Clients  located  at  overseas  installations  did  prefer  to  contact 
the  NETWORK  by  mail. 

Responses  to  user  inquiries  were  prepared  generally  within  a  five  working 
day  time  period  as  anticipated  by  the  NETWORK  staff.  However,  one-third  of 
the  client  inquiries  exceeded  the  anticipated  response  time  period.  Users 
evaluated  the  NETWORK'S  response  turnaround  time  as  more  than  satisfactory. 
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The  materials  provided  by  the  NETWORK  in  response  to  information  requests 
related  to  a  variety  of  topics  and  required  various  formats.  Users  were 
more  than  satisfied  with  the  materials  provided  to  them  in  response  to 
their  requests.  These  responses  were  judged  to  be  relevant  to  and  useful 
in  meeting  the  users'  information  needs.  In  addition,  users  tended  to 
share  the  response  materials  with  their  colleagues.  Finally,  the  materials 
provided  to  the  NETWORK'S  clients  were  utilized  in  applications  associated 
with  administrative  planning/review  activities  and  counseling/testing 
tasks.  It  was  surprising  to  note  that  response  materials  were  not  used  in 
relation  to  teaching  or  training  activities;  however,  some  clients  were 
resourceful  in  using  the  NETWORK'S  materials  for  in-service  training  and 
professional  development  activities. 

Generally,  all  clients  who  utilized  the  NETWORK'S  Inquiry  Response  and 
Referral  Services  stated  that  they  would  continue  to  use  these  services  and 
would  recommend  them  to  others.  However,  the  NETWORK'S  user  groups  were 
basically  not  sure  how  often  they  would  require  direct  assistance  from  the 
NETWORK. 

Military  educational  personnel  were  very  positive  about  the  quality  and 
usefulness  of  the  NETWORK'S  publications.  All  publications  were  rated  very 
high  in  terms  of  comprehensiveness,  ease  of  understanding,  accuracy  of 
information,  and  format.  The  NETWORK'S  publications  were  considered  to  be 
an  above  average  mechanism  through  which  military  educators  were  able  to 
identify  resources,  stay  updated  on  topics,  and  follow  current  research 
efforts. 
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Initial  pilot  testing  of  the  NETWORK  Profiles  Service  indicated  that  this 
specialized  information  service  had  moderate  potential.  One-half  of  the 
pilot  test  participants  stated  that  they  were  satisfied  with  the 
information  materials  provided  to  them  and  found  these  materials  to  be 
useful  to  their  professional  activities.  Finally,  a  majority  of  the 
participants  noted  that  the  Profiles  Service  should  be  continued  and  would 
be  useful  to  their  colleagues. 

Generation  of  a  computerized  database  was  undertaken  to  support  the 
information  services  offered  by  the  NETWORK.  Overall,  computer  usage  and 
staff  charges  associated  with  the  development  of  the  NETWORK'S  database 
remained  within  projected  expenditures. 

A  variety  of  subject  areas  were  indicated  to  be  of  interest  to  military 
educational  personnel.  Information  categories  of  special  interest  were: 
basic  skills  curricula,  computer-assisted  capabilities  and  evaluations; 
career  planning  and  guidance;  computer-based  guidance  systems;  general 
management  skills,  needs  assessment  techniques;  and  program  and  curriculum 
evaluations. 

Finally,  a  set  of  fifteen  recommendations  are  offered  to  guide  the  future 
operation  of  the  NETWORK.  These  recommendations  capitalize  on  the 
network's  proven  success  and  suggest  minor  modifications  to  the  information 
services  offered  through  the  NETWORK  in  order  to  better  meet  the 
professional  development  and  information  needs  of  military  educators. 
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I.  INTRODUCTION 


The  Basic  Skills  Resource  Center  (BSRC),  developed  and  operated  by 
I nter Amer I ca  Research  Associates,  Inc.,  was  initiated  in  April  1982  under 
Contract* Number  MDA  903-82-C-0I69  with  the  U.S.  Army  Research  Institute  for 
the  Behavioral  and  Social  Sciences  (ARI).  The  BSRC  consists  of  two 
interfacing  components:  a  research  component  that  includes  the  design, 
implementation,  and  coordination  of  a  learning  strategies  research  agenda; 
and  an  information  component  that  involves  the  design  and  operation  of  an 
information  service  entitled  the  "Military  Educators  Resource  NETWORK." 
This  report  provides  a  final  summary  of  the  activities  related  to  the  BSRC 
information  component.  In  addition,  the  information  contained  in  this 
final  report  provides  a  synthesis  of  the  results  of  the  evaluative  data 
solicited  throughout  the  initial  operational  phase  of  the  NETWORK. 

This  report  is  the  last  In  a  series  of  five  reports  which  describe  project 
activities  as  well  as  results  and  recommendations  related  to  the  BSRC 
information  component.  The  first  report  (see  Russo,  Rivera,  DeCarme,  & 
French,  in  press-a)  described  the  purpose  and  results  of  a  needs  assessment 

I 

undertaken  to  focus  the  developmental  efforts  associated  with  the 
information  service.  The  second  report  (see  Rivera,  Russo,  S  DeCarme,  in 
press-b)  delineated  an  operational  design  plan  for  the  information  services 
to  be  offered  by  the  Military  Educators  Resource  NETWORK.  An  interim 
report  (see  Russo,  In  press-c)  was  prepared  as  the  third  report  and 
provided  a  description  of  the  status  of  the  NETWORK  activities  five  months 
after  the  design  plan  was  implemented.  The  fourth  report  (see  Russo, 
Foster,  &  Modjeski,  in  press-d)  detailed  the  NETWORK'S  operational 
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order  to  facilitate  the  continued  operation  of  the  NETWORK'S  services. 
This  report,  the  fifth  in  the  series,  serves  as  the  final  report  of  the 
activities  associated  with  the  BSRC  information  component. 


Purpose  and  Objectives  of  the  NETWORK'S  Evaluation  Activities 


The  Military  Educators  Resource  NETWORK  has  been  in  full  operation  for 
seventeen  months,  from  March  1983  through  July  198A.  This  operational 
period  has  provided  the  NETWORK  staff  with  the  opportunity  to  pilot  test 
the  design  plan  specified  and  recommended  by  project  staff  associated  with 
the  BSRC.  The  design  plan,  implemented  by  the  NETWORK  staff,  was  a  result 
of  a  needs  assessment  conducted  by  InterAmerica  staff  and  the  refinement  of 
a  procedural  plan  reviewed  by  InterAmerica  and  ARI  staff  as  well  as  by 
personnel  within  the  Department  of  the  Army's  Adjutant  General's  Office 
(TAGO). 


Throughout  the  seventeen  month  operational  period,  a  variety  of  internal 
and  external  evaluative  data  have  been  gathered  for  the  purpose(s)  of 
describing  the  NETWORK'S  operational  procedures,  perceived  quality  and  use 
of  services  provided  by  the  NETWORK,  and  perceived  quality  of  informational 
publications  developed  by  the  NETWORK  staff.  These  purposes  are  achieved 
through  the  accomplishment  of  the  following  objectives: 


Describe  the  NETWORK'S  user  population  in  terms  of  the 
educators  t  i  1 1 e/ pos i t i on  and  military  service 
af f i 1 i at  ion. 

Describe  the  method  used  to  submit  information  requests, 
as  well  as  response  formats  and  the  preparation  time 
(turnaround  time)  for  responses  to  inquiries  logged  by 
the  NETWORK. 

Describe  the  degree  of  satisfaction  to  information 
responses  provided  by  the  NETWORK. 
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o  Describe  the  perceived  quality  of  information  products/ 
publications  developed  and  disseminated  by  the  NETWORK. 

o  Describe  the  perceived  usefulness  of  the  NETWORK'S 
specialized  information  services. 

o  Describe  and  synthesize  associated  costs  related  to  the 
development  and  operation  of  the  NETWORK'S  computerized 
database. 

o  Describe  the  perceived  information  needs,  and 
anticipated  usage  rates  relative  to  the  NETWORK'S  target 
population. 

Evaluative  data  were  acquired  through  activities  such  as  the  maintenance  of 
a  user  activity  log  and  the  distribution  of  questionnaires  and  checklists 
related  to  the  NETWORK'S  services.  This  report  provides  a  synthesis  of  the 
evaluative  data  maintained  by  the  NETWORK  and  identifies  a  set  of 
recommendations  for  the  future  operation  of  the  NETWORK'S  information 
services. 

The  remaining  portions  of  this  chapter  provide  an  overview  of  the  results 
of  needs  assessment  activities  and  a  description  of  the  operational  design 
plan  established  for  the  Military  Educators  Resource  NETWORK.  Included 
with  this  background  information  is  a  description  of  the  overriding  purpose 
established  for  the  NETWORK  and  identification  of  the  purpose  and 
objectives  of  each  service  offered  by  the  NETWORK. 

Chapter  Two  summarizes  the  results  of  data  collection  efforts  pertaining  to 
each  of  the  objectives  outlined  above.  In  addition,  brief  descriptions  of 
the  evaluation  activities  and/or  questionnaires,  as  well  as  data  collection 
procedures,  are  provided.  Finally,  Chapter  Three  provides  a  set  of 
recommendations  that  should  be  considered  in  the  future  operation  of  the 
NETWORK. 
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Prior  to  the  actual  provision  of  a  set  of  information  services  for  military 
educators,  BSRC  project  staff  were  required  to  identify  the  scope  and 
function  of  the  BSRC  information  component.  In  addition,  an  operational 
plan  for  the  information  component  needed  to  be  specified.  These  tasks 
were  accomplished  through  the  completion  of  a  needs  assessment  and 
development  of  a  design  plan.  Each  of  these  activities  is  discussed  below. 


Needs  Assessment.  The  first  activity  undertaken  by  InterAmerica  project 
staff  related  to  the  BSRC  information  component  was  a  needs  assessment 
study  that  was  designed  to  provide  a  pool  of  information  that  would  assist 
project  staff  in  the  development  of  an  information  service  for  military 
educators.  Specifically,  the  following  objectives  were  addressed: 

o  to  determine  who  would  be  the  major  users  of  the 

information  service, 

o  to  assess  the  information  needs  of  potential  user 

groups, 

o  to  identify  the  scope  of  the  database  to  be  developed, 

o  to  identify  the  services  that  should  be  made  available, 

and 

o  to  determine  how  information  should  be  made  available  to 
users. 

Information  required  to  address  each  of  these  objectives  was  collected 
between  April  and  June  I982.  Data  collection  activities  included: 
distribution  of  a  questionnaire  targeted  for  Education  Services  Officers 
(ESOs)  and  Education  Services  Specialists  (ESSs);  telephone  and  in-person 
interviews  with  Army  educators/practitioners,  researchers  and  policymakers; 
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site  visits  to  two  Major  Commands  (MACOMs),  four  installation  education 
centers,  and  three  military  i ibraries/learning  centers;  and  review  of  Army 
regulations  and  documents.  These  data  were  then  synthesized  by  project 
staff  so  that  each  of  the  specific  objectives  could  be  addressed. 

The  needs  assessment  findings  revealed  that  the  target  groups  for  the  BSRC 
information  service  would  be  practitioners,  researchers,  and  policymakers 
within  the  Department  of  the  Army.  The  greatest  proportion  of  potential 
users  was  found  to  be  the  Army's  practitioner/educator  group  which  includes 
ESOs,  ESSs,  counseiors,  and  other  education-related  personnel  at  Army 
instal lations. 

/ 

No  specific  trends  were  identified  with  regard  to  major  subject  areas  to  be 
addressed  by  the  information  system.  Potential  users  identified  a  wide 
range  of  topics.  These  included:  counseling  information  related  to  career 
planning  and  guidance  and  computer-based  guidance  systems;  education 
information  related  to  basic  skills  curricula,  instructional  materials  and 
tests  as  well  as  computer-assisted  instruction;  management  information 
related  to  contracting  requirements,  needs  assessment  techniques,  program 
and  curriculum  evaluation  techniques  and  research  methods;  and  general 
military  information  regarding  demographic  data,  and  research  and 
programmatic  efforts  in  operation  at  other  military  installations. 

Useful  formats  for  this  information  cited  by  the  potential  user  groups 
included  abstracts  of  individual  documents,  bibliographies,  curriculum  and 
learning  materials,  literature  searches,  newsletters,  referral  services, 
research  summaries,  and  statistical  data.  Potential  users  provided  little 
guidance  in  estimating  the  frequency  of  use  of  an  information  service. 


Generally,  it  was  reported  that  the  information  center  would  be  used 
approximately  once  a  month. 

The  preferred  method  of  accessing  the  information  center  was  found  to  be  by 
computer,  with  telephone  access  considered  an  acceptable  alternative.  All 
potential  users  indicated  a  need  for  prompt  turnaround  time  for  the  receipt 
of  information  once  a  request  was  submitted.  The  most  frequently 
acceptable  turnaround  time  period  was  stated  to  be  three  to  four  days. 
Based  on  these  findings,  several  recommendations  were  noted  regarding 
design  considerations  for  the  BSRC  information  center.  First,  the  center 
should  include  a  computerized  information  database  and  inquiry  response 
system  that  could  actively  reach  out  to  users  and  supply  them  with 
information  based  on  specific  as  well  as  anticipated  requests.  Second,  the 
services  to  be  offered  by  the  information  center  should  be  targeted  to 
educators/pfactitioners  associated  with  Army  education  programs.  Third, 
the  focus  of  services  and  subject  areas  to  be  addressed  by  the  information 
center  should  be  limited  to  those  areas  identified  by  the  primary  target 
group.  Fourth,  the  services  offered  by  the  information  center  should 
include  a  proactive  component  to  encourage  and  stimulate  user  requests. 
Finally,  telephone  access  should  be  the  primary  mode  of  accessing  the 
information  center,  and  an  established  schedule  of  expected  response 
turnaround  times  should  be  identified. 

Operational  Design  Plan.  Following  the  completion  of  the  needs  assessment 
activities,  project  staff  began  the  development  of  an  operational  design 
plan  for  the  BSRC  information  center  to  be  referred  to  as  the  Military 
Educators  Resource  NETWORK.  Through  a  review  of  the  needs  assessment  data 
and  resulting  design  considerations  as  well  as  subsequent  discussions  with 
ARI  and  The  Adjutant  General's  Office  (TAGO)  staff,  an  overriding  mission 
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or  purpose  was  detailed  for  the  NETWORK.  The  stated  purpose  for  the 
NETWORK  is:  to  assist  the  Army  in  disseminating  up-to-date  information 
relevant  to  basic  skills  and  continuing  education  issues  to  educational 
practitioners,  researchers,  and  policymakers  within  the  Department  of  the 
Army.  The  design  plan  (see  Rivera,  Russo,  S  DeCarme,  in  press-b)  provided 
an  operational  framework  that  outlined  the  purposes  and  objectives  of  the 
NETWORK,  the  primary  and  secondary  user  groups,  the  services  to  be 
provided,  and  the  content  and  focus  of  the  services.  Three  basic  functions 
were  identified  to  carry  out  the  mission  established  for  the  NETWORK. 
These  included:  (a)  the  development  of  a  computerized  database;  (b)  the 
dissemination  of  information  through  the  provision  of  the  Inquiry  Response 
and  Referral  Services,  a  Publication  Development  and  Dissemination  Service, 
and  a  Current  Awareness  Service;  and  (c)  the  evaluation  of  these  services 
relative  to  target  group  usage  and  reactions  during  the  initial 
implementation  period. 

Implementat ion.  The  NETWORK  design  plan  recommended  by  InterAmerica  staff 
was  implemented  in  March  I983  which  marked  the  beginning  of  the  formal 
pilot  test  of  the  NETWORK'S  services.  At  this  time  project  staff  also 
formalized  the  development  of  the  NETWORK'S  computerized  database.  The 
computerized  database  provides  the  NETWORK  staff  with  the  capability  and 
information  to  respond  to  user  requests  via  the  inquiry  response  service 
and  to  provide  poi nts-of-contact  or  referrals  to  relevant  individuals  and 
organizations.  A  complete  description  of  the  NETWORK'S  database  is 
provided  in  an  earlier  report  entitled  "Documentation  and  Phaseover  Report 
for  the  Military  Educators  Resourcd  NETWORK"  (see  Russo,  Foster,  and 
Modjeski,  in  press-c).  As  noted,  the  database  provides  support  to  the 
information  services  offered  by  the  NETWORK.  Each  of  these  services  is 
discussed  below  in  terms  of  its  overall  purpose  and  goals. 


The  Inquiry  Response  Service  is  designed  to  operate  as  a  reference  service 
assisting  users  in  the  identification  and  location  of  information.  Using 
the  network's  database  and  library  collection  as  well  as  related 
information  sources,  abstracts  and  citations,  resources  are  identified  and 
provided  to  users  in  response  to  their  inquiries. 


The  Referrai  Service  is  intended  to  provide  the  user  with  the  name  of  an 
individual  or  organization  that  would  most  likely  be  able  to  respond  to  the 
user's  request  or  provide  additional  detailed  information.  Generally, 
referral  information  relates  to  on-going  military  research  activities  and 
educational  programs. 


The  Publication  Development  and  Dissemination  Service  is  planned  to  provide 


a  mechanism  that  allows  for  the  proactive  dissemination  of  information  in 
the  area  of  adult  and  continuing  education  and  serves  to  stimulate  user 
requests  through  the  promotion  of  NETWORK  services.  These  objectives  are 
achieved  primarily  through  the  development  and  distribution  of  a  quarterly 
newsletter  entitled  the  NETWORK  Circuit  and  a  fact  sheet  that  is 
intermittently  disseminated  entitled  the  NETWORK  Fact  Sheet. 


The  Current  Awareness  Service  is  designed  to  provide  a  link  between  the 


network's  Inquiry  Response  and  Referral  Services  and  the  Publication 
Development  and  Dissemination  Service.  This  service  disseminates 
information  to  users  at  periodic  intervals  based  on  projected  or 
pre-identi f ied  user  interests.  There  are  two  current  awareness  activities 
that  distribute,  on  a  regular  basis,  information  about  new  resources  or 
advances  in  the  adult  basic  skills  and  continuing  education  field.  These 
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activities  are  (a)  periodic  distribution  of  the  NETWORK  Vanguard  which 
contains  photocopies  of  the  tables-of-contents  of  key  journals  in  the 
education  field  and  (b)  the  NETWORK  Profile  Service  which  provides 
information  resources  to  a  designated  set  of  Army  educators  according  to 
their  pre-stated  information  interests. 

Each  of  these  services  and  associated  activities  are  discussed  in  greater 
detail  in  the  NETWORK  Design  Plan  (see  Rivera,  Russo,  S  DeCarme,  in 
press-b).  In  addition  to  the  development  of  a  computerized  database  and 
the  offering  of  an  integrated  set  of  information  services,  the  NETWORK'S 
design  plan  specified  an  evaluation  function  intended  to  monitor  the 
effectiveness  of  the  NETWORK'S  activities  during  its  initial  operational  or 
pilot  test  period.  The  specific  evaluation  activities  designed  and 
implemented  by  project  staff  are  discussed  in  detail  in  the  next  section  of 
this  report. 
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II.  EVALUATION  RESULTS 


An  important  function  of  the  NETWORK  is  the  collection  and  maintenance  of 
evaluative  information  related  to  the  NETWORK'S  activities.  The  basic 
intent  of  the  NETWORK'S  evaluation  is  to  determine  the  overall 
effectiveness  of  the  NETWORK'S  information  services  and  procedures  in 
meeting  its  specified  purpose  and  objectives.  This  was  achieved  through  a 
series  of  evaluation  activities  that  were  planned  by  project  staff  to 
assess  the  NETWORK  during  its  initial  operational  or  pilot  test  phase.  The 
evaluation  activities  were  designed  to  study  the  NETWORK'S  services  from 
both  an  internal  and  external  perspective  and  to  address  a  series  of 
operational  questions.  The  evaluation  activities  and  key  questions  are 
highlighted  in  Figure  A. 

This  section  of  the  final  report  summarizes  the  responses  to  and  results  of 
the  various  evaluation  activities  completed  by  the  NETWORK  staff. 
Summaries  are  presented  for  each  of  the  following  services  and/or 
components:  (a)  Inquiry  Response  and  Referral  Service;  (b)  Publication 
Development  and  Dissemination  Service;  (c)  Current  Awareness  Service;  and 
(d)  Computerized  Database  Development.  The  evaluative  information 
discussed  in  each  of  the  summaries  identifies  the  key  questions  that  the 
evaluation  activities  addressed.  In  addition,  a  brief  overview  describing 
relevant  instruments  and  maintenance  and/or  distribution  procedures  is 
provided.  Finally,  when  appropriate,  response  rates  are  cited. 


Together,  these  summaries  are  designed  to  provide  a  set  of  indicators  by 
which  the  effectiveness  and  success  of  the  information  services  offered  by 


the  NETWORK  can  be  judged.  This  set  of  indicators  also  provides  direction 
to  the  identification  of  recommendations  for  the  future  operation  of  the 
NETWORK.  These  recommendations  are  presented  and  discussed  in  the  final 
section  of  this  report. 


A.  Inouirv  Resoonse  and  Referral  Services 


These  NETWORK  services  provide  direct  assistance  to  users  through  the 
identification  and/or  location  of  information  relevant  to  the  users' 
information  requests.  Several  evaluation  activities  were  conducted  to 
obtain  evaluative  information  about  these  services.  These  activities 
included  the  maintenance  of  operational  information  as  well  as  the 
identification  of  user  reactions  to  NETWORK  procedures  relevant  to 
information  provided  through  these  services.  Summaries  of  both  the 
operational  data  and  user  reaction  data  are  presented  below. 


Operational  Information 


Two  internal  evaluation  activities  (see  Figure  A)  were  implemented  by 
project  staff  that  were  designed  to  gather  operational  data  about  the 
Inquiry  Response  and  Referral  Services.  These  activities  were  undertaken 
to  address  the  following  key  questions: 


Who  were  the  users  of  these  services? 

How  frequently  did  clients  use  these  services? 

What  method  did  clients  use  to  contact  the  NETWORK? 

What  type(s)  of  information  was  (were)  provided  to  users  in 
response  to  their  request? 

What  was  the  response  time  for  the  information  requests  submitted 
to  the  NETWORK? 
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The  operational  information  needed  to  address  each  of  these  questions  was 
maintained  by  NETWORK  staff  throughout  the  pilot  test  period  through  the 
use  of  an  Information  Request  Log  and  an  Information  Request  Form.  Project 
staff  utilized  the  Request  Log,  exhibited  in  Appendix  A,  to  record  user 
requests  in  the  order  they  were  received  via  telephone,  mail  or  in  person. 
The  Request  Log  provided  a  mechanism  by  which  information  inquiries  were 
monitored  and  enabled  staff  to  maintain  basic  statistical  data  about  the 
Inquiry  Response  and  Referral  Services.  The  Information  Request  Form, 
exhibited  in  Appendix  B,  was  used  by  the  NETWORK  clients  and/or  staff  to 
articulate  and  record  user  inquiries.  The  Request  Form  elicits  descriptive 
information  related  to  the  inquiry  (e.g.,  key  concepts  related  to  the 
user's  information  needs,  type  of  Information  user  is  interested  in, 
required  response  format),  basic  demographic  information  about  the  client, 
and  action  taken  by  staff  in  the  preparation  of  a  response  to  the  inquiry. 


Together,  the  operational  information  maintained  through  the  use  of  these 
internal  documentation  procedures  allows  data  summaries  to  be  prepared  in 
response  to  the  questions  cited  above.  Evaluative  information  related  to 
the  operational  questions  associated  with  the  Inquiry  Response  and  Referral 
Services  is  presented  in  Tables  1  through  6  and  is  discussed  below. 


The  network's  Clients.  A  variety  of  military  educational  personnel 
utilized  the  NETWORK'S  services  throughout  the  initial  pilot  test  phase. 
Included  is  a  practitioner  group  which  includes  Education  Services  Officers 
(ESOs),  Education  Services  Specialists  (ESSs),  Guidance  Counselors  and 
various  education-related  staff,  for  example,  users  who  title  themselves 
interns,  training  officers,  and  testing  officers.  The  practitioner 
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category  is  considered  the  primary  user  group  since  the  NETWORK  services 
were  targeted  specifically  for  these  individuals.  A  second  major  group 
includes  administrators  which  represent  command  and  headquarters  staff  such 
as  the  MACOM  Directors  of  Education  and  TAGO  staff.  Finally,  a  third  group 
of  users  represents  researchers  such  as  AR)  personnel.  (Note:  These  user 
group  categories  are  used  throughout  the  remaining  discussion  presented  in 
this  report.) 

Tabie  1  exhibits  the  number  of  information  requests  recorded  by  month  of 
operation  and  by  type  of  client.  Overall,  a  total  of  532  information 
requests  were  responded  to  over  the  NETWORK'S  seventeen  month  operational 
period.  The  practitioner  group  (ESOs  -  23^;  ESSs  -  l4|;  guidance 
counselors  -  16%;  and  education-related  staff  -  18%)  accounted  for  71%  of 
the  overall  information  inquiries  while  the  administrator  group  and  the 
researcher  group  accounted  for  12%  and  3%  respectively.  The  client 
category  entitled  "others"  accounted  for  8%  of  the  information  requests  and 
included  military  education  personnel  such  as  recruiters  and  installation 
i ibrar ians. 

Reviewing  the  monthly  request  totals,  the  following  benchmarks  can  be 
identified.  The  March  and  April  1983  requests  coincided  with  the 
distribution  of  the  NETWORK'S  first  publication,  the  NETWORK  Brochure  and 
Rolodex  Card.  Requests  for  May,  June,  and  July  1983  were  to  be  stimulated 
by  the  distribution  of  the  first  newsletter  and  fact  sheet.  However,  due 
to  the  Department  of  the  Army's  extremely  lengthy  review  and  approval 
cycles  associated  with  these  publications,  they  were  not  disseminated  until 
August  1983.  Thus,  the  monthly  totals  for  August  and  September  1983 
reflect  the  impact  of  the  distribution  of  these  initial  publications  on 
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user  contacts.  A  slight  decline  was  noted  in  October  I983  due  to  the 
lengthy  review  and  approval  cycles  inhibiting  the  dissemination  of  the 
second  issues  of  the  newsletter  and  fact  sheet.  Requests  for  the  months  of 
November  1983i  December  1983i  January  198A,  and  February  1984  were  again 
stimulated  by  the  distribution  of  the  NETWORK'S  publications  and  the 
inclusion  of  these  publications  with  the  DANTES  newsletter. 

The  time  period  of  March  1984  through  July  1984  exhibited  a  continued 
decline  in  total  requests  due  primarily  to  the  fact  that  the  initial 
operational  phase  for  the  NETWORK  was  advertised  to  conclude  in  February 
1984.  The  operational  timeframe  was  extended  through  July  1984  because  of 
the  delays  experienced  in  the  review  and  approval  of  the  NETWORK'S 
publications.  Throughout  this  period,  the  remaining  issues  of  the 
network's  publications  were  disseminated  both  by  the  NETWORK  staff  and  in 
coordination  with  the  DANTES  newsletter. 

Table  2  presents  monthly  information  request  totals  by  the  military  service 
affiliation  of  the  NETWORK'S  clients.  Overall,  of  the  NETWORK'S  user 
population  were  affiliated  with  the  Department  of  the  Army.  The  remaining 
one-third  included  users  associated  with  the  Air  Force  -  141;  the 
Navy  -  7%,  the  Coast  Guard  -  4%;  and  the  Marines  -  7^.  The  category 
referenced  as  "other"  contains  7%  of  the  client  population  and  includes 
users  affiliated  with  the  Department  of  Defense  and  clients  associated  with 
military  education  and  training  programs.  As  exhibited  in  Table  2,  the 
Department  of  the  Army  was  the  primary  target  population  of  the  NETWORK'S 
information  services  from  March  1983  through  October  1983*  In  November 
1983  and  throughout  the  remaining  operational  period,  the  information 
services  were  made  available  to  all  military  service  branches.  This  was 
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accomplished  with  the  dissemination  of  information  about  the  NETWORK  and 
the  inclusion  of  NETWORK  publications  with  monthly  issues  of  the  DANTES 
newsletter.  However,  throughout  the  pilot  test  period,  the  Department  of 
the  Army  represented  at  least  50%  of  the  NETWORK'S  monthly  clients. 

Frequency  of  Use.  A  summary  of  the  number  of  Information  requests 
submitted  to  the  NETWORK  by  the  requestor  title/ posit  ion  is  presented  in 
Table  3.  Overall,  80%  (303/381)  of  the  NETWORK'S  clients  utilized  the 
network's  services  only  once.  The  remaining  20%  were  Identified  as  repeat 
users  of  the  information  services.  Specifically,  27%  (20/73)  of  the 
Education  Services  Officers;  25%  (14/58)  of  the  Education  Services 
Specialists;  and  24%  (14/59)  of  the  guidance  counselors  contacted  the 
NETWORK  with  two  or  more  information  requests.  Overall,  only  7%  of  the 
network's  clients  were  considered  Researchers. 

Method  of  Contact .  Table  4  provides  a  summary  of  the  method  by  which 
users  contacted  the  NETWORK.  Throughout  the  NETWORK'S  pilot  test  period, 
three  methods  of  inquiry  were  available.  These  were:  telephone,  mail,  and 
answering  machine.  Access  to  an  answering  machine  service  was  provided  in 
order  to  faci I i tate  ,use  of  the  NETWORK'S  services  by  individuals  assigned 
to  overseas  locations  and  with  work  hours  other  than  between  9:00  a.m.  and 
4:30  p.m.  eastern  time. 

The  primary  method  of  contact  was  by  mail  (55%).  Users  also  contacted  the 
NETWORK  by  telephone  (43%).  The  preferred  method  of  contact  varied  monthly 
between  mail  and  telephone  and  exhibited  no  logical  pattern.  Only  one 
information  request  was  logged  using  the  answering  machine  indicating  that 
this  method  is  not  a  viable  approach  for  receiving  information  inquiries. 
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Finally,  a  small  percentage  {7%)  of  requests  was  recorded  through  in-person 
visits  to  the  NETWORK  offices  or  at  professional  conferences  where  the 
NETWORK  offered  an  exhibit.  These  latter  methods  of  contact  are  recorded 
in  the  "other"  category  in  Table  4. 

Information  Responses  to  User  Requests.  The  response  formats  related  to 
the  information  inquiries  logged  by  the  NETWORK  are  exhibited  in  Table  5. 
Depending  upon  the  user's  information  request,  a  unique  response  was 
prepared  by  the  NETWORK  staff  and  transmitted  to  a  requestor.  The  format 
of  response  could  include:  citations  of  information  resources  related  to 
the  inquiry;  points-of-contact  to  individuals  and/or  organizations;  related 
materials  such  as  newsletters,  conference  announcements,  and  vendor 
pamphlets;  explanatory  material  about  the  NETWORK'S  services;  or  an 
addition  or  update  to  the  NETWORK'S  mailing  list.  A  user's  inquiry  may 
involve  one  or  more  of  these  categories  or  formats  of  responses.  Monthly 
totals  for  the  response  formats  are  exhibited  in  Table  5.  Overall, 
responses  to  inquiries  required  information  citations  approximately  30^  of 
the  time  and  required  points-of-contact  and  related  materials  approximately 
20?  of  the  time.  In  addition,  roughly  25%  of  the  responses  required  an 
update  of  the  NETWORK'S  mailing  list. 

Inspection  of  the  monthly  totals  reveals,  as  can  be  expected,  that  the 
"mailing  list  update"  and  "information  on  the  NETWORK"  formats  were  highest 
during  periods  where  the  NETWORK'S  services  were  highly  publicized.  These 
were:  March  and  April  I983  following  the  distribution  of  the  NETWORK'S 
Brochure  and  Rolodex  Card;  August  and  September  I983  following 
dissemination  of  the  first  issues  of  the  newsletter  and  fact  sheet; 
November  and  December  1983  following  the  initial  incorporation  of  materials 


about  the  NETWORK  in  the  DANTES  newsletter,  and  May  1984  following  the 
dissemination  of  the  DANTES  newsletter  to  the  Recruiting  Commands. 
In  total,  inquiries  for  additional  "information  on  the  NETWORK"  accounted 
for  8%  of  the  response  formats.  This  small  percentage  rate  may  indicate 
that  the  explanatory  materials  about  the  NETWORK'S  services  included  in  the 
network's  publications  were  useful  to  users  in  conveying  the  goals  and 
functions  of  the  NETWORK. 

Response  Turnaround  Time.  The  NETWORK  staff,  in  recognition  of  the  need 
for  a  prompt  turnaround  time  to  user  inquiries,  have  established  a  policy 
of  responding  to  requests  within  three  to  five  working  days.  Monthly 
request  response  times  for  the  seventeen  month  pilot  test  phase  are 
exhibited  in  Table  6.  Overall,  6?^  of  ail  requests  were  responded  to  in 
five  working  days  or  less.  Approximately  one-third  (34:^)  of  all  requests 
were  responded  to  the  same  day  the  inquiry  was  logged  by  the  NETWORK  staff. 
These  latter  requests  often  required  response  formats  for 
po i nt s-of -contact ,  NETWORK  materials,  and/or  mailing  list  updates. 
Responses  requiring  numerous  database  searches  and  the  identification  of 
supplemental  information  materials  took  longer  than  five  working  days  to 
prepare,  and  occurred  with  33?  of  the  responses.  The  only  trend  apparent 
in  the  preparation  of  responses  as  summarized  in  Table  6  is  that  the 
greater  the  number  of  requests  the  NETWORK  logged  the  longer  the  response 
turnaround  time. 

It  should  be  noted  that  the  request-response  time  summary  was  based  on  the 
availability  of  one  full-time  Information  Specialist  assigned  to  the 
NETWORK.  Such  a  staffing  pattern  proved  sufficient  to  respond  effectively 
to  all  requests  requiring  response  formats  for  points-of-contact ,  NETWORK 


materials  and/or  mailing  list  updates  at  the  projected  level  or  load  of 
requests  recorded  during  the  NETWORK'S  operational  cycle.  In  addition  to 
this  load,  a  fuH-time  Information  Specialist  would  be  able  to  effectively 
process  60?  to  70?  of  the  information  requests  requiring  response  formats 
of  detailed  database  searches  and  the  identification  of  supplemental 
materials.  Thus,  an  additional  Information  Specialist  or  support  staff 
person  would  be  required  to  respond  efficiently  to  al  1  requests  within 
three  to  five  working  days  for  peak  request  levels  as  experienced  during 
the  pilot  test  phase. 

User  Reactions 


The  Inquiry  Response  and  Referral  Services  operate  as  a  reference  service 
to  assist  users  in  the  identification  and  location  of  information  as  well 
as  points-of-contacts  relevant  to  the  user  information  needs.  Depending 
upon  the  user's  information  request,  a  response  is  prepared  and  transmitted 
to  the  requestor.  As  previously  noted,  responses  to  requests  can  involve 
one  or  more  formats  of  information.  These  include:  computerized  database 
searches;  poi nts-of-contact  to  individuals  and/or  organizations;  and 
related  materials  such  as  newsletters,  conference  announcements,  and  vendor 
pamphlets. 

In  order  to  solicit  information  about  user  reaction  to  the  Inquiry  Response 
and  Referral  Services,  the  NETWORK  conducted  three  related  evaluation 
activities.  Together,  these  activities  allowed  judgements  to  be  made 
regarding  the  following  major  questions: 
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o  Were  users  satisfied  with  the  response  turnaround  time? 

o  Were  the  NETWORK'S  clients  satisfied  with  the  informa¬ 
tion  provided  to  them  in  response  to  their  request? 

o  Do  users  plan  to  continue  to  utilize  the  NETWORK'S 
services? 

In  order  to  address  these  questions,  information  was  acquired  through  the 
three  activities  discussed  below.  First,  project  staff  used  a  brief, 
six-item  instrument  designed  to  identify  the  degree  of  user  satisfaction 
with  the  results  of  computerized  database  searches  conducted  to  locate 
information  citations  pertinent  to  the  user's  information  needs.  The 
NETWORK  Computer  Search  Evaluation  Form  (see  Appendix  C)  was  included  with 
the  search  results  provided  to  the  inquirer  in  response  to  their  request. 
Participation  in  this  evaluation  activity  was  strictly  voluntary  and 
respondents  remained  anonymous.  A  total  of  48  evaluation  forms  were 
distributed  with  al  1  search  results  prepared  in  response  to  inquiries 
logged  during  the  time  period  from  February  1984  through  July  1984. 
However,  users  who  submitted  multiple  requests  during  this  time  period  were 
provided  with  only  one  copy  of  the  Search  Form  in  order  to  avoid  response 
duplication.  The  NETWORK'S  staff  received  a  total  of  26  completed 
evaluation  forms  yielding  a  response  rate  of  54?.  These  responses  are 
summarized  in  Table  1. 

Second,  a  set  of  telephone  interviews  was  conducted  to  assist  in 
determining  user  satisfaction  with  information  responses  prepared  to  meet 
their  information  needs.  Roughly  every  twentieth  military  educator  who 
used  the  Inquiry  Response  or  Referral  Service  was  asked  to  participate  in  a 
brief  telephone  interview.  During  the  telephone  call  the  educator  was 
asked  to  respond  to  six  questions  (see  Appendix  D)  that  focused  on  their 
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satisfaction  with  the  information  response  provided  by  the  NETWORK  to  their 
inquiry. 

Teiephone  caiis  were  made  to  a  total  of  10  users  during  July  1983  in 
preparation  for  the  NETWORK'S  Interim  Report  and  a  total  of  20  additional 
users  during  August  1984  in  preparation  for  the  NETWORK'S  Final  Report.  A 
summary  of  responses  to  the  teiephone  calls  is  exhibited  in  Table  8. 

Finally,  project  staff  developed  a  structured  evaluation  instrument 
designed  to  determine  the  effectiveness  of  the  Inquiry  Response  and 
Referral  Services  as  judged  by  the  users  of  these  services.  This  brief 
thirteen  item  instrument,  entitled  the  NETWORK  User  Questionnaire  (see 
Appendix  E),  was  disseminated  with  the  third  issue  of  the  NETWORK  Fact 
Sheet.  This  questionnaire  was  distributed  to  approximately  750  military 
educators  listed  on  the  NETWORK'S  mailing  list.  In  addition,  copies  of  the 
questionnaire  were  disseminated  with  a  DANTES  newsletter.  A  total  of  191 
completed  questionnaires  were  voluntarily  returned  to  the  NETWORK.  An 
accurate  response  rate  cannot  be  determined  because  (1)  respondents 
remained  anonymous  and  were  encouraged  to  duplicate  the  NETWORK'S  materials 
for  use  by  colleagues,  and  (2)  the  total  number  of  questionnaires 
distributed  by  DANTES  is  unknown. 

The  military  service  affiliation  and  the  title/position  of  the  respondents 
to  the  NETWORK  User  Questionnaire  are  summarized  in  Table  9.  Overall,  48^ 
of  the  returned  questionnaires  were  received  from  educators  associated  with 
the  Department  of  the  Army.  In  addition,  241  were  received  from  educators 
affiliated  with  the  Navy  and  15^  from  Air  Force  related  educators.  The 
Education  Service  Officers  were  the  largest  group  of  respondents  for  each 
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Telephone  Interviews- -Response  Results 


of  the  service  branches  and  accounted  for  42%  (n-81)  of  the  respondents 
overall.  The  Education  Service  Specialists  and  Guidance  Counselors 
accounted  for  19%  and  18%  of  the  respondents,  respectively. 

User  Satisfaction  with  Response  Turnaround  Time.  The  NETWORK  User 
Questionnaire  contained  a  series  of  items  related  to  how  often  the 
respondent  used  the  NETWORK'S  services,  the  NETWORK'S  response  time  to  the 
requests,  and  the  respondent's  satisfaction  with  the  speed  of  response. 
The  responses  to  these  items  are  exhibited  in  Table  10.  Overall,  66%  of 
the  respondents  indicated  that  they  had  not  submitted  an  information 
request  to  the  NETWORK.  Responses  to  this  item  ranged  from  37%  for  Army 
educators  to  100%  for  Marine  educators.  Those  educators  who  had  used  the 
NETWORK'S  services  indicated  that  primarily  they  had  submitted  at  least  one 
request  (20%;  n>38)  and  sometimes  as  many  as  two  to  'four  requests  (9%; 
n«l8).  This  request  pattern  is  consistent  across  the  service  branches. 

Overall,  84%  of  the  respondents  who  used  these  services  indicated  that 
their  request(s)  was  responded  to  within  three  weeks  following  the 
submission  of  their  information  request.  Respondents  (72%)  rated  their 
satisfaction  with  the  speed  of  response  as  "more  than  acceptable"  to 
"extremely  good."  Overall,  the  speed  of  response  was  not  rated  lower  than 
acceptable  by  any  of  the  respondents,  with  only  6%  of  the  respondents 
rating  response  turnaround  time  as  "slightly  less  than  acceptable." 

These  findings  were  confirmed  by  the  evaluative  information  acquired  via 
the  Computer  Search  Evaluation  Form  and  the  Telephone  Interviews. 
Responses  to  the  Search  Evaluation  Form  (see  Table  7)  indicated  that  al  1 
respondents  judged  the  response  time  to  their  requests  as  "average"  to 
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"excellent."  The  response  time  to  requests  was  rated  "excellent"  by  62^  of 
the  respondents.  In  addition,  63%  of  the  NETWORK'S  clients  who 
participated  in  the  telephone  interviews  Judged  the  response  time  to 
requests  as  "excellent."  Only  3%  of  the  educators  interviewed  indicated 
that  the  turnaround  time  was  "poor"  (see  Table  8). 

User  Satisfaction  with  Information  Response.  The  NETWORK  User 
Questionnaire  also  contained  a  set  of  items  which  required  respondents  to 
rate  their  satisfaction  with  the  information  provided  by  the  NETWORK  in 
response  to  their  inquiry,  to  indicate  how  the  information  was  used,  to 
specify  if  the  information  was  shared  with  others,  to  Judge  the  ease  of 
using  the  information,  and  to  identify  the  appropriateness  of  the  referral 
information.  A  summary  of  responses  to  these  items  is  exhibited  in  Table 
11.  Overall,  respondents  indicated  that  the  information  materials  were 
useful  primarily  for  administrative  planning/review  activities  (48%), 
counseling/  testing  activities  (40%),  and  research  activities  (40%). 
Surprisingly,  few  educators  (5%)  indicated  that  the  information  materials 
were  used  in  association  with  teaching  or  training  activities. 

Army  educators  indicated  that  the  information  was  used  for  administrative 
planning/ review  (51%)  as  well  as  counseling/testing  (49%)  activities. 
These  trends  are  not  as  apparent  in  the  responses  from  educators  associated 
with  the  other  service  branches.  Responses  categorized  as  "other"  indicate 
that  both  Army  and  Air  Force  educators  were  creative  in  the  use  of 
information  materials  in  their  in-service  training  and  professional 
development  activities. 

Primarily,  respondents  (76%)  rated  their  satisfaction  with  the  information 
materials  provided  to  them  as  "acceptable"  to  "extremely  good."  Respondent 
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satisfaction  with  the  materials  was  also  demonstrated  by  their  indication 
that  the  materials  were  shared  with  others.  A  total  of  711  of  the  military 
educators  shared  the  NETWORK  information  with  others.  Respondents  were 
also  asked  to  indicate  the  number  of  persons  with  whom  the  materials  were 
shared.  Sixty-three  percent  of  the  respondents  indicated  that  they  shared 
the  network's  materials  with  one  to  as  many  as  six  other  individuals.  The 
format  and  organization  of  materials  provided  to  inquirers  were  judged  to 
be  "easy  to  use"  by  82i  of  the  respondents.  Finally,  the  points-of -contact 
or  referrals  provided  in  response  to  user  information  requests  were  cited 
by  of  the  respondents  as  being  "appropriate"  to  their  request.  Only  8? 
of  the  respondents  indicated  that  the  referral  information  was 
"i nappropr i ate." 

Information  identified  through  the  Computer  Search  Evaluation  Form  and 
Telephone  Interview  confirmed  the  above  findings.  Specifically,  853!  of  the 
respondents  to  the  Search  Evaluation  Form  (see  Table  7)  indicated  they  were 
satisfied  with  the  results  of  the  customized  search  and  that  the  number  of 
information  citations  identified  was  sufficient.  In  addition,  6l%  of  the 
respondents  indicated  that  50  to  100  percent  of  the  information  citations 
identified  for  them  were  relevant  to  their  information  needs. 

NETWORK  clients  who  participated  in  the  telephone  interviews  (see  Table  8) 
judged  the  information  materials  provided  by  the  NETWORK  in  response  to  the 
individual's  requests  as  being  "useful"  (50%)  to  "extremely  useful"  (A0%). 
Overall,  the  information  provided  to  inquirers  "met  the  expectations"  of 
7^%  of  the  respondents.  Additionally,  23%  of  the  educators  stated  that  the 
materials  "exceeded  their  expectations." 
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Anticipated  Use  of  the  Inquiry  Response  and  Referral  Services.  A  third 
and  final  set  of  items  included  in  the  NETWORK'S  User  Questionnaire  asked 
users  to  indicate  if  they  would  recommend  the  NETWORK'S  services  to  others 
and  if  they  planned  to  use  the  NETWORK'S  services  again  and  if  so,  how 
often.  Responses  to  these  items  are  summarized  and  presented  in  Table  12. 
Ninety-four  percent  of  the  respondents  indicated  that  they  would  use  the 
network's  services  again.  Primarily,  respondents  were  not  sure  how  often 
they  would  use  the  information  services,  although  29i  of  the  educators 
indicated  that  services  would  be  used  six  or  more  times  during  a  calendar 
year  and  26^  indicated  that  they  would  use  the  NETWORK  less  than  six  times 
during  a  calendar  year.  No  predictable  patterns  were  noted  for  respondents 
affiliated  with  the  different  service  branches.  In  addition,  853!  of  the 
respondents  indicated  that  they  would  recommend  the  NETWORK'S  services  to 
others. 

As  in  prior  cases,  the  evaluation  information  collected  through  the 
Computer  Search  Evaluation  Form  and  the  Telephone  Interview  served  to 
confirm  users'  intentions  about  the  continued  use  of  the  NETWORK.  A1 I 
respondents  to  the  search  evaluation  form  (see  Table  7)  indicated  that  they 
received  satisfactory  assistance  from  the  NETWORK  staff  and  that  they 
planned  to  utilize  the  NETWORK'S  services  again.  In  addition,  a1  1 
participants  in  the  telephone  Interviews  (see  Table  8)  stated  that  the 
NETWORK  staff  were  "very  helpful"  (93^)  or  "helpful"  (7^)  in  assisting 
users  with  their  information  needs.  Finally,  al  1  respondents  indicated 
that  they  planned  to  use  the  NETWORK  in  the  future. 
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This  NETWORK  service  was  designed  to  distribute  to  military  educators 
information  about  the  NETWORK'S  services  and  information  related  to  current 
topics  in  adult  and  continuing  education.  This  goal  was  achieved  through 
the  preparation  of  publications  entitled  the  NETWORK  Circuit,  the  NETWORK 
Fact  Sheet,  and  the  NETWORK  Vanguard.  (Note:  This  latter  publication  is 
considered  a  component  of  the  Current  Awareness  Service,  but  is  discussed 
in  this  section  of  the  final  report  since  evaluative  information  about  all 
publications  was  coordinated  through  one  evaluation  instrument).  In  order 
to  assess  these  publications,  project  staff  developed  an  evaluation 
instrument  that  would  address  the  following  key  questions: 

o  How  helpful  are  the  publications  to  the  NETWORK'S 

users? 

o  What  is  the  perceived  quality  of  the  NETWORK'S 

pubi icat ions? 

o  What  is  the  perceived  usefulness  of  the  NETWORK'S 

publ icat ions? 

Users'  perceptions  were  identified  through  the  distribution  of  the  NETWORK 
Reaction  Sheet  (see  Appendix  F)  which  was  distributed  to  approximately  750 
military  educators  listed  on  the  NETWORK'S  mailing  list.  In  addition, 
copies  of  the  instrument  were  disseminated  with  a  DANTES  newsletter.  A 
total  of  125  completed  instruments  were  voiuntarily  returned  to  the 
NETWORK.  An  accurate  response  response  rate  cannot  be  determined  because 
(1)  respondents  remained  anonymous  and  were  encouraged  to  duplicate  the 
network's  materials  for  use  by  colleagues,  and  (2)  the  total  number  of 
questionnaires  distributed  by  DANTES  is  unknown. 
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The  military  service  affiliation  and  the  title/position  of  the  respondents 
to  the  NETWORK  Reaction  Sheet  are  summarized  in  Table  13>  Overall,  621  of 
the  returned  questionnaires  were  received  from  Army  education  personnel. 
In  addition,  16?  were  received  from  educators  associated  with  the  Navy  and 
10%  with  Air  Force  affiliated  personnel.  Education  Services  Officers  and 
Education  Services  Specialists  accounted  for  353!  and  25%  of  the  responding 
population  respectively.  Overall,  these  two  groups  of  respondents 
accounted  for  60%  of  the  completed  instruments. 

Helpfulness.  The  evaluation  instrument  requested  the  NETWORK'S  target 
population  to  indicate  the  general  level  of  helpfulness  of  the  three 
publications  on  a  five-point  scale  defined  as:  very  helpful  ("5"), 
moderately  helpful  "3"}  or  "2"),  and  not  helpful  ("1").  The  rating 
scale  included  a  no  opinion/not  received  ("0")  response  option.  In 
addition,  respondents  were  asked  to  rate  their  satisfaction  with  the 
quality  and  usefulness  of  the  NETWORK'S  publications.  The  following 
five-point  scale  was  used:  extremely  good  ("5"),  acceptable  ("4",  "3",  or 
"2"),  and  very  poor  ("1").  This  rating  scale  also  included  a  no  opinion 
("0")  response  option.  Table  14  exhibits  the  mean  ratings  of  the 
helpfulness,  quality,  and  usefulness  of  the  NETWORK'S  publications  by  the 
respondents'  title/position.  Three  segments  of  the  respondent  population 
are  presented  (i.e.,  ESOs,  ESSs,  and  Guidance  Counselors)  because  these 
groups  represent  60%  of  the  response  ratings.  The  "other"  category 
combines  the  responses  received  from  researchers,  administrators,  and 
others.  The  three  educator  groups  responded  very  similarly  in  their  rating 
of  the  helpfulness  of  the  NETWORK  Circuit,  Fact  Sheet,  and  Vanguard.  All 
groups  indicated  that  these  publications  were  "more  than  moderately 
helpful"  as  indicated  by  mean  ratings  which  ranged  from  3.5  to  3*8. 
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Qual i ty .  The  quality  of  the  NETWORK'S  publications  was  judged  in  terms  of 
comprehensiveness,  ease  of  understanding,  accuracy  of  information,  and 
format  based  on  the  following  five-point  scale:  extremely  good  ("5"), 
acceptable  ("4",  "3",  or  "2")  and  very  poor  {"1").  Each  of  these  factors 
was  rated  moderately  below  "extremely  good"  as  indicated  by  mean  ratings 
ranging  from  4.0  to  4.3.  Mean  ratings  representative  of  the  various 
titles/positions  were  fairly  consistent. 

Usefulness.  The  NETWORK'S  publications  were  also  judged  by  respondents 
relative  to  their  usefulness  which  was  defined  in  the  following  terms: 
usefulness  as  a  means  of  learning  about  a  topic,  value  in  identifying 
resources  for  military  educators,  and  overall  usefulness  in  the  educator's 
work.  Respondents  were  asked  to  use  the  following  five-point  scale: 
extremely  good  ("5"),  acceptable  ("4",  "3",  or  "2")  and  very  poor  ("1"). 
Overall,  the  mean  rating  for  these  factors  was  judged  to  be  "more  than 
acceptable"  as  exhibited  by  the  mean  ratings  ranging  from  3»4  to  3«9* 

The  NETWORK  Reaction  Sheet  also  asked  respondents  to  indicate  the  various 
ways  the  publications  were  used  and  to  indicate  if  the  publications  were 
shared  with  their  colieagues.  The  responses  provided  to  these  items  are 
summarized  in  Table  15.  Responses  are  summarized  by  three  primary 
respondent  groups,  that  is,  ESOs,  ESSs,  and  Guidance  Counselors.  All 
groups  indicated  that  the  information  was  used  primarily  to  obtain  an 
update  on  topics  and  to  identify  resources.  ESOs  also  used  the 
publications  to  follow  current  research  efforts.  In  addition,  all  educator 
groups  strongly  indicated  that  the  NETWORK  publications  were  shared  with 
thei r  col  leagues. 
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ponstis  to  the  NETWORK  Reaction  Sheet  by  Respondents'  Title/Position 
by  Items  Related  to  the  Use  of  the  NETWORK'S  Publications 


C.  Current  Awareness  Service 


As  noted  earlier,  this  information  service  is  composed  of  two  activities. 
First,  the  preparation  and  distribution  of  the  NETWORK  Vanguard,  containing 
photocopies  of  professional  journal  tables-of-contents,  is  designed  to 
increase  military  educators'  awareness  of  current  articles,  book  reviews, 
etc.  User  reactions  regarding  the  usefulness,  quality,  and  helpfulness  of 
this  NETWORK  publication  were  identified  through  the  NETWORK  Reaction 
Sheet,  Responses  to  this  evaluation  instrument  were  presented  and 
summarized  in  the  preceding  section  (see  Tables  14  and  15). 

A  second  current  awareness  activity  involved  the  pilot  testing  of  the 
NETWORK  Profile  Service.  A  small  group  (n-19)  of  overseas  Army  educators 
were  asked  to  participate  in  the  initial  offering  of  this  specialized 
service.  Fourteen  of  the  nineteen  military  educators  who  elected  to 
participate  were  provided  special  sets  of  information  materials  that 
related  to  their  pre-stated  information  interests  profile.  Participants 
were  asked  to  complete  a  brief  questionnaire  designed  to  identify  their 
satisfaction  with  the  information  materials  and  to  determine  the  usefulness 
of  this  specialized  service.  This  questionnaire,  entitled  the  NETWORK 
Profile  Service  Opinion  Form  (see  Appendix  G),  was  completed  by  79^  (n*11) 
of  the  participants.  The  voluntary  responses  to  the  Opinion  Form  are 
summarized  in  Table  16. 

Participants  in  the  pilot  testing  of  the  NETWORK  Profile  Service  indicated 
that  they  were  generally  "satisfied”  (64%)  with  the  information  materials 
provided  to  them;  however,  18%  of  the  educators  indicated  that  they  were 
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"not  satisfied"  with  the  materials.  About  one-haif  (55%)  of  the 
participants  indicated  that  the  information  materiais  were  "usefui"  to 
their  professionai  activities  as  compared  to  36%  of  the  respondents  who 
indicated  the  materiais  were  "not  useful."  As  a  related  indicator  to  the 
usefulness  of  materials  and  the  participant's  satisfaction  with  this 
specialized  service,  respondents  were  asked  to  indicate  if  the  information 
materiais  provided  to  them  were  shared  with  their  colleagues.  The  majority 
(73%)  of  the  participants  indicated  that  they  did  share  the  materials  with 
thei r  coi leagues. 

Finally,  participants  were  asked  to  indicate  the  usefulness  of  this  service 
to  other  military  educators  and  to  recommend  whether  or  not  the  NETWORK 
should  continue  to  offer  the  Profile  Service.  In  both  instances, 
approximately  two-thirds  (6A%)  of  the  respondents  indicated  that  this 
specialized  service  would  be  "useful"  to  their  colleagues  and  that  the 
NETWORK  should  continue  to  offer  the  Profile  Service. 

D.  Computerized  Database  Development 

The  design  plan  developed  for  the  NETWORK  specified  the  formation  of  a 
computerized  database  that  would  support  the  various  information  services 
offered  by  the  NETWORK.  As  identified  through  the  needs  assessment 
activities,  the  coi lection  of  materials  contained  in  the  database  wouid 
include  information  representative  of  programmatic  and  research  efforts  in 
basic  sk i 1 i s  education.  In  addition,  the  coilection  wouid  include 
citations  of  resources  describing  Army  basic  skilis  education  programs  as 
weil  as  reference  and  referrai  information.  During  the  NETWORK'S  initial 
operational  phase,  project  staff  developed  a  computerized  database  that 
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would  meet  the  information  needs  expressed  by  military  educators.  This 
section  of  the  final  report  addresses  the  following  key  questions: 


o  What  kind  and  type  of  information  is  contained  in  the 
network's  database? 

o  What  costs  are  associated  with  the  development  and 
operation  of  the  NETWORK'S  database? 

o  What  subject  areas  are  of  interest  to  the  NETWORK'S 
users? 

Database  Description.  Tables  17  and  l8  provide  summary  descriptions  of 
the  database  in  terms  of  publication/document  type  and  key  descriptors. 
The  majority  of  the  citations  included  in  the  database  describe  resources 
that  are  of  the  following  types:  Reports  (436),  Journal  Articles  (257), 
and  Guides  (220).  In  addition,  the  major  key  descriptors  assigned  to  the 
citations  maintained  in  the  NETWORK'S  database  include:  Computer-Assisted 
Instruction  (183),  Instructional  Materials  (l4l),  Teaching  Methods  (108), 
Military  Training  (98),  Reading  Instruction  (97),  and  Program  Evaluation 
(94). 

Development  and  Operation  Costs.  A  computerized  database  facilitates  the 
standardization  of  the  information  storage  and  retrieval  processes  needed 
to  provide  information  to  users.  Development  of  a  computer-based 
information  system  can  potentially  be  a  high  resource  expenditure.  Thus, 
costs  associated  with  the  NETWORK'S  database  developmental  tasks  were 
monitored  throughout  the  pilot  test  period. 

Computer  costs  monitored  for  this  period  include  computer  staff  time 
charges  incurred  during  the  construction  of  the  in-house  NETWORK  database 
as  well  as  fees  associated  with  the  use  of  the  NETWORK  and  commercially 
available  databases  to  respond  to  inquiries. 


TABLE  17 

NETWORK  DATABASE  CITATIONS  BY  PUBLICATION/DOCUMENT  TYPE 


PUBLICATION/DOCUMENT  TYPE 

TOTAL 

Books 

37 

Col lected  Works 

32 

Oi ssertat ions/Theses 

5 

Gu i des 

220 

Historical  Materials 

13 

Information  Analyses  (Literature  Reviews,  State-of-the-Art  Papers) 

53 

Journal  Articles 

257 

Leg  is lative/Regulatory  Materials 

4 

Statistical  Data 

12 

Viewpoints  (Opinion  Papers,  Position  Papers,  Essays,  etc.) 

96 

Reference  Materials 

67 

Reports 

436 

Speeches,  Conference  Papers 

81 

Tests,  Evaluation  Instruments 

27 

Other/Mi  seel laneous 

9 

GRAND  TOTAL 

1 , 349* 

The  costs  associated  with  computer  personnel  apply  to  the  design, 
development,  and  implementation  phase  of  the  NETWORK  database  and  are 
exhibited  in  Table  19.  Monthly,  yearly  and  total  costs  for  computer 
personnel  are  listed  in  this  table  for  the  nineteen  month  period  from 
February  1983  to  July  198^.  These  charges  reflect  personnel  expenditures 
for  computer  programmers  and  systems  analysts  as  well  as  data  entry  staff. 
For  the  nineteen  month  developmental  period,  a  total  of  1,219  staff  hours 
were  expended  for  an  associated  total  direct  cost  (excluding  fringe 
benefits)  of  $7 » ^20. 00. 

In  the  initial  phase  of  the  operation  of  the  project,  a  large  part  of 
computer  staff  time  was  devoted  to  database  design  and  development 
involving  such  activities  as  formatting  records  for  input,  report  design, 
and  internal  database  construction  within  the  InterAmerica  Digital 
Equipment  Corporation  (DEC)  system.  Formatting  records  for  input  involved 
the  generation  of  forms  for  both  NETWORK  database  files  which  provided  the 
pertinent  information  to  be  included  in  each  record  to  be  added  to  the 
database  (see  Russo,  Foster,  and  Modjeski,  in  press-c).  Report  design  by 
various  computer  staff  produced  printed  output  in  the  form  of  an 
easy-to-read  citation  which  would  be  forwarded  in  response  to  information 
requests.  These  citations  were  designed  in  two  distinct  forms  reflecting 
the  information  contained  in  each  NETWORK  file  (see  Russo,  Foster,  and 
Modjeski,  in  press-c)  and  depending  upon  the  type  of  information  requested. 
Internal  database  construction  involved  programmers  who  adapted  the 
specifications  of  the  NETWORK  database  to  be  compatible  with  the  Data 
Retrieval  System  'DRS)  software  package  used  with  the  DEC  system.  This 
initial  phase  occurred  from  February  to  August  1983  and  accounted  for 
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approximately  1S%  of  the  staff  time  and  staff  charges  associated  with  the 
network's  database. 

Data  entry  staff  were  used  throughout  the  project  beginning  in  February 
1983  until  completion  of  the  initial  construction  phase  in  July  198A.  Data 
entry  staff  duties  included  original  input,  edited  input,  and  other 
functions  necessary  for  the  construction  of  the  database.  Data  entry  staff 
also  provided  the  support  needed  for  report  generation  to  produce  necessary 
reports  and  print  out  citations  in  the  special  NETWORK  formats.  The 
monthly,  yearly  and  total  costs  for  computer  personnel  are  listed  in  this 
table.  Also  included  in  the  table  are  the  monthly  costs  of  indexers  who 
periodically  assisted  the  Information  Specialist  with  the  preparation  of 
records  for  inclusion  into  the  database. 

Total  costs  for  computer  connect  time,  both  in-house  and  on  commercially 
available  databases,  are  listed  in  Table  20.  Included  in  the  figures  for 
the  InterAmerica  DEC  system  is  the  time  used  by  the  Information  Specialist 
for  searching  the  NETWORK  database  for  pertinent  citations  to  respond  to 
inquiries.  Also  included  in  the  InterAmerica  connect  time  charges  are  the 
computer  usage  costs  for  the  time  data  entry  staff  input  new  records  or 
edited  previous  input.  Additional  computer  time  was  incurred  while  various 
reports  were  generated.  Monthly  charges  for  searching  external  commercial 
databases  are  also  presented  in  Table  20.  These  latter  computer  usage 
charges  consisted  of  communication  costs,  royalties,  individual  database 
connect  time  charges,  and  offline  printing  fees. 

Over  the  nineteen  month  period,  in-house  computer  usage  totaled 
approximately  758  connect  hours  reflecting  an  associated  cost  of  $10,512.00 
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In  addition,  external  computer  usage  resulted  in  a  direct  cost  of 
$3)520.00,  roughly  25%  of  the  total  net  computer  usage  costs  ($14,032.00). 

The  seven  month  initial  developmental  period  for  February  I983  through 
August  1983  accounted  for  approximately  47%  of  the  total  internal  connect 
charges.  The  remaining  months  and  related  charges  reflect  the  operational 
period  associated  with  the  maintenance  of  the  database  in  support  of  the 
network's  information  services.  Computer  usage  activities  during  this 
time  frame  included  editing,  inputting,  report  generation,  and  direct 
searching  of  the  database. 

/ 

User  Information  Interest.  A  specialized  survey  was  conducted  by  project 
staff  to  assist  in  the  generation  of  the  NETWORK'S  computerized  database. 
A  Subject  Area  Checklist  (see  Appendix  H)  was  designed  to  provide  NETWORK 
staff  with  information  to  detefmine  which  subject  areas  are  most  important 
and/or  of  the  greatest  interest  to  military  educators.  Based  on  the 
results  of  this  survey,  citations  in  these  subject  areas  were  incorporated 
into  the  NETWORK  database  as  it  was  expanded  during  the  construction  phase. 
Results  from  this  survey  can  also  be  used  to  provide  guidance  for  future 
growth  of  the  database. 

The  Subject  Area  Checklist  was  distributed  to  approximately  65O  military 
educators  listed  on  the  NETWORK'S  mailing  list.  in  addition,  copies  of  the 
instrument  were  distributed  with  a  DANTES  newsletter.  A  total  of  172 
survey  forms  were  returned  to  the  NETWORK.  An  accurate  response  rate 
cannot  be  determined  because  (1)  respondents  remained  anonymous  and  were 
encouraged  to  duplicate  the  NETWORK'S  materials  for  use  by  colleagues,  and 
(2)  the  total  number  of  checklists  distributed  by  DANTES  is  unknown. 


5 


Table  21  presents  a  summary  of  the  respondents  to  the  Subject  Area 
Checklist  by  title  or  position  within  each  of  the  military  services. 
Overall,  65^  of  the  respondents  were  ESSs  {}(>%)  and  ESOs  {2S%) .  In 
addition,  6?^  of  the  respondents  were  military  educators  associated  with 
the  Department  of  the  Army.  Educators  affiliated  with  the  Air  Force  (57%) 
and  Coast  Guard  (50^)  represented  a  majority  of  the  ESO  respondent  category 
as  compared  to  the  Navy  (73%)  and  Army  (30%)  which  represented  a  majority 
of  the  ESS  respondent  category.  Counselors  comprised  another  23%  of  the 
total  respondent  group  while  "Researchers"  and  "Others"  only  accounted  for 
12%  of  the  total  number  of  respondents. 

The  responses  to  the  Subject  Area  Checklist  are  presented  in  Table  22. 
Responses  are  summarized  by  the  military  service  affiliations  of 
respondents  and  by  subject  areas  in  five  major  categories:  Education, 
Counseling,  Computer  Systems,  Government/Military  Information,  and 
Management.  Respondents  were  asked  to  check  all  the  areas  in  which 
information  would  be  useful  to  them  in  the  operation  of  their  ACES 
programs.  Each  individual  subject  area  listed  in  Table  22  denotes  the 
number  of  times  an  area  was  checked  and  the  percentage  this  number 
represents  in  regard  to  the  total  number  of  respondents. 

The  category  with  the  most  subject  areas  is  that  of  Education.  Significant 
totals  were  recorded  in  the  Basic  Skills  Curricula,  Computer-Assisted 
Instruction,  and  Reading  Skills  areas.  In  each  case,  approximately  50%  of 
the  total  number  of  respondents  chose  these  areas.  In  all  three  areas,  the 
Army  and  Marines  were  significantly  higher  than  the  other  three  services. 
Of  secondary  interest  to  educators  were  the  areas  of  Functional  Skills, 
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Percentages  may  add  to  more  than  100%  since 
respondents  checked  "all  that  applied." 


Percentages  may  add  to  more  than  100%  since 
respondents  checked  "all  that  applied." 


Quality  Assurance,  Self-Paced  Instructional  Programs,  and  Tests  and 
Measurements.  Consistently  low  percentages  of  M%  and  l8^  were  recorded  in 
the  areas  of  Psychomotor  and  Daily  Life  Coping  Skills. 

In  the  category  of  Counseling,  the  primary  areas  of  interest  to  respondents 
were  Career  Planning  and  Guidance  iS3%)  and  Information  on  Colleges  (52%). 
Secondary  areas  of  interest  were  Computer-Based  Guidance  Systems  (47%)  and 
Civilian  Labor  Force  Market  Data  (47%).  Overall,  Air  Force,  Army,  and  Navy 
personnel  recorded  higher  total  percentages,  indicating  a  higher  interest 
in  counseling  than  Coast  Guard  or  Marine  personnel. 

Each  of  the  subject  areas  included  in  the  Computer  Systems  category  was 
rated  very  highly,  indicating  avid  interest  in  this  category.  However,  in 
view  of  the  summary  of  responses  by  military  service,  much  disparity  is 
apparent.  The  percentage  of  responses  provided  by  Air  Force  and  Navy 
personnel  ranged  from  0%  to  23%.  Educators  associated  with  the  Army,  Coast 
Guard  and  Marines  provided  a  range  of  response  percentages  from  50%  to  79%. 
High  total  percentages  were  recorded  in  the  areas  of  Computer  Equipment 
(52%)  and  Software  (58%)  Evaluations. 

In  the  Government/Military  Information  category,  none  of  the  subject  areas 
showed  high  percentages  with  the  exception  of  "Research  Programs  -  Army"  by 
the  Army  group  (60%)  and  "Research  Programs  -  Other  Military"  by  the  Coast 
Guard  at  50%,  Marines  at  100?,  and  Navy  at  58%.  These  findings  may  suggest 
that  educators  are  interested  only  in  information  about  research  that  is 
supported  by  their  military  service  branch.  Low  percentages  were  noted  in 
the  areas  of  Federal  Budget  Appropriations  (22%),  Directives  and  Regulatory 
Information  (26%),  and  Demographic  Data  (28%). 


The  subject  area  of  Needs  Assessment  Techniques  had  the  highest  percentage 
of  responses  both  in  the  Management  category  and  in  overall  responses.  One 
hundred  fourteen  respondents  constituting  S0%  of  the  total  population 
indicated  an  interest  in  this  subject  with  the  Army  at  70?  and  the  Navy  at 
69?.  Also  in  the  Management  category,  the  areas  of  General  Management 
Skills  (A2?)  and  Program  and  Curriculum  Evaluation  (48?)  were  significant 
while  the  least  amount  of  interest  was  exhibited  in  Research  Methods  (22?). 
High  interest  was  indicated  by  the  Navy  (65?)  in  Marketing  Education 
programs  and  the  Army  showed  high  interest  (53?)  in  Contracting 
Requirements.  Overall,  respondents  indicated  a  high  degree  of  interest  for 
most  of  the  subject  areas  cited  in  the  Management  category. 

Findings  associated  with  the  internal  and  external  evaluation  activities 
were  identified  in  this  section  of  the  final  report.  Discussions  of  these 
findings  and  their  implications  relative  to  the  future  operation  of  the 
NETWORK  are  presented  in  the  following  section. 
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III.  DISCUSSION  AND  RECOHMENDATIONS 


One  of  the  three  basic  functions  established  for  the  operation  of  the 
NETWORK  was  the  completion  of  a  set  of  evaluation  activities  related  to  the 
assessment  of  the  NETWORK.  Nine  evaluation  activities  were  undertaken  to 
study  the  NETWORK  from  both  an  internal  and  external  perspective.  The 
information  services  were  judged  In  terms  of  the  extent  to  which  the 
services  were  used  by  the  NETWORK'S  targeted  user  groups  and  the  users' 
perceptions  of  the  overall  usefulness  and  quality  of  services  during  the 
initial  operational  period  established  for  the  NETWORK.  This  section  of 
the  report  includes  a  discussion  of  the  evaluation  results  presented  in  the 
preceding  section.  In  addition,  a  set  of  recommendations  relative  to  the 
future  operation  of  the  NETWORK  is  presented  for  each  of  the  following 
information  components:  Inquiry  Response  and  Referral  Services, 
Publication  Development  and  Dissemination  Service,  Current  Awareness 
Service,  and  Computerized  Database  Development. 

A.  Inquiry  Response  and  Referral  Services 

The  primary  users  of  these  two  NETWORK  services  were  education 
practitioners.  This  group  of  military  educators  includes  Education 
Services  Officers  (ESOs),  Education  Services  Specialists  (ESSs),  counselors 
and  education-related  staff.  Overall,  ESOs  utilized  these  services  most 
frequently.  In  addition,  practitioners  were  the  most  frequent  repeat 
users.  Practitioners  had  been  the  specific  group  of  educators  expected  to 
have  the  most  contact  with  the  NETWORK.  As  indicated  by  the  needs 
assessment  results,  it  was  anticipated  that  researchers  and  administrators 
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respectively  would  follow  the  practitioner  group  in  terms  of  overali 
inquiries  for  information.  However,  the  administrator  group  utilized  the 
network's  services  more  frequently  than  the  researcher  group.  Based  on  the 
total  number  of  requests,  the  NETWORK  provided  the  most  services  to 
practitioners  and  program  administrators  who  had  direct  contact  with  the 
educational  programs  offered  by  the  military  services. 

As  would  be  expected,  educators  affiliated  with  the  Department  of  the  Army 
accounted  for  a  majority  of  the  NETWORK'S  user  population  throughout  the 
initial  operational  period.  Once  access  to  the  information  services  was 
provided  to  non-Army  service  branches.  Air  Force  personnel  proved  to  be  the 
second  most  frequent  group  of  users.  Findings  suggest  that  during  the 
future  operation  of  the  NETWORK,  at  least  one-half  of  the  user  population 
would  be  affiliated  with  the  Department  of  the  Army  and  would  represent 
practitioners  associated  with  the  military  educational  programs.  Based  on 
these  findings  the  following  recommendations  are  offered: 

Recommendat ion  1 ;  Continue  to  focus  the  NETWORK'S  information 
services  for  the  educator  groups  directly  associated  with  the 
military  educational  programs.  Specifically,  ESOs,  ESSs, 
counselors,  and  program  administrators. 

Recommendation  2;  Continue  to  publicize  the  availability  of  the 
network's  services  to  Department  of  the  Army  educators  and  expand 
publicity  about  the  NETWORK  among  educators  affiliated  with  the 
Departments  of  the  Air  Force,  Coast  Guard,  Marines,  and  Navy. 

The  network's  clients  preferred  to  contact  the  NETWORK  with  their 
information  requests  via  mail  and  telephone.  Although  the  majority  of 
inquiries  were  submitted  by  mail  rather  than  by  telephone,  no  set  pattern 
or  preference  was  clearly  indicated.  However,  written  communication  was 
chosen  primarily  by  educators  located  outside  the  continental  United 
States.  The  availability  of  an  answering  machine  for  logging  requests 
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proved  not  to  be  viable.  Since  this  option  was  provided  to  meet  the 
special  needs  of  educators  located  overseas,  alternative  modes  of  contact 
need  to  be  explored.  Based  on  the  user  contact  patterns  noted  during  the 
operational  phase,  the  following  recommendations  are  offered: 


Recommendation  3^  Continue  to  provide  clients  with  access  to  the 
network's  information  services  via  telephone  and  mail.  A 
dedicated  telephone  number  and  mailing  address,  possibly  a  Post 
Office  Box  Number,  should  be  established  and  maintained. 

Recommendation  Terminate  access  to  the  NETWORK'S  services  via 
a  telephone  answering  machine.  The  feasibility  of  alternative 
modes  of  contact  should  be  explored  such  as  the  establishment  of 
a  toll-free  "800"  telephone  number,  or  the  utilization  of  an 
Autovon  and/or  Federal  Telephone  System  number.  The  alternatives 
should  be  considerate  of  the  special  needs  of  overseas  military 
educators. 


Information  requests  recorded  by  the  NETWORK  staff  required  the  preparation 
of  a  variety  of  responses.  On  the  average,  one-half  of  the  information 
inquiries  required  responses  consisting  of  computerized  database  searches 
and/or  the  identification  of  po ints-of -con tact .  The  remaining  inquiries 
included  requests  for  information  about  the  NETWORK'S  services,  updates  to 
the  network's  mailing  list,  and  requests  for  information  materials  such  as 
the  network's  publications.  Responses  were  prepared  as  quickly  and 
efficiently  as  possible.  The  NETWORK  staff  responded  to  two-thirds  of  all 
requests  within  five  working  days.  However,  detailed  requests  and  peak 
inquiry  periods  extended  the  NETWORK'S  response  turnaround  time.  Feedback 
provided  by  the  NETWORK'S  clients  indicated  that  the  response  turnaround  to 
inquiries  was  more  than  acceptable.  Therefore,  the  following 
recommendation  is  offered: 


Recommendation  5?  Maintain  the  policy  of  preparing  and 
transmitting  responses  to  inquiries  within  three  to  five  working 
days.  Clients  submitting  detailed  inquiries  should  be  informed 
that  a  response  to  their  request  will  require  additional  time  for 
preparation. 


The  response  formats  to  information  requests  included  information  citations 
identified  through  database  searches,  information  materials  and  resources 
as  well  as  referrals  to  individuals  and  professional  organizations.  A  vast 
majority  of  the  users  of  these  information  services  indicated  that  they 
were  more  than  satisfied  with  the  information  the  NETWORK  provided  in 
response  to  their  inquiry.  In  addition,  a  vast  majority  of  the  NETWORK'S 
clients  judged  the  materials  to  be  relevant  to  their  information  needs. 
Finally,  a  majority  of  clients  indicated  that  the  information  materials 
were  acceptable  and  useful  for  their  needs  and  that  they  often  shared  the 
materials  with  their  colleagues. 

The  network's  primary  user  groups  indicated  that  the  materials  provided  to 
them  by  the  NETWORK  was  primarily  used  for  activities  associated  with 
administrative  planning/review  and  counseling/testing  tasks.  This  trend 
was  most  apparent  with  educators  affiliated  with  the  Department  of  the 
Army.  Overall,  very  few  military  educators  indicated  that  the  materials 
provided  to  them  were  used  in  relation  to  teaching  or  training  activities. 
This  finding  may  suggest  that  military  educators  who  work  with  "contracted" 
personnel  responsible  for  direct  instruction  and/or  training  did  not  share 
the  network's  information  materials  with  these  colleagues.  Because  the 
feasibility  of  allowing  contract  personnel  access  to  the  NETWORK'S  services 
during  the  pilot  test  phase  remained  ambiguous,  this  potential  group  of 
users  was  not  actively  solicited  as  a  target  audience.  If  military 
educators  remain  the  sole  target  audience,  the  NETWORK  will  enhance 
activities  (e.g.,  program  planning/review,  counseling/testing)  that 
indirectly  impact  on  the  actual  teaching  or  training  environment. 
Expanding  the  user  group  to  include  "contract"  teachers  and/or  trainers  may 
result  in  a  more  direct  impact  on  teaching  and  training  activities. 
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Finally,  a  few  educators  proved  to  be  very  resourceful  In  the  application 
of  materials  provided  to  them  by  the  NETWORK.  Both  Army  and  Air  Force 
clients  indicated  that  the  materials  were  useful  for  in-service  training 
and  professional  development  activities.  In  consideration  of  these 
findings,  the  following  recommendation  is  offered; 

Recommendation  6;  Determine  the  feasibility  of  expanding  the 
NETWORK  target  groups  to  include  teachers  and  trainers  associated 
with  military  educational  programs.  If  appropriate,  publicity 
about  the  availability  of  the  NETWORK'S  services  should  be 
expanded  and  targeted  to  teachers  and  trainers  in  order  to 
facilitate  the  NETWORK'S  impact  on  teaching  and  training 
activities. 

Generally,  all  clients  who  used  the  NETWORK'S  Inquiry  Response  and  Referral 
Services  indicated  that  they  would  use  these  services  again.  In  addition, 
users  of  these  services  stated  that  they  would  recommend  these  services  to 
others.  These  findings  suggest  that  once  an  educator  becomes  an  active 
participant  in  the  services  offered  by  the  NETWORK  they  plan  to  become 
repeat  users  and  will  encourage  others  to  become  actively  involved  in  the 
NETWORK.  These  findings  also  suggest  that  the  NETWORK  staff  have  been  able 
to  convey  to  the  NETWORK'S  clients  an  accurate  description  of  services  and 
results  to  be  expected  through  use  of  the  NETWORK'S  information  services. 

The  network's  targeted  user  groups  were  basically  not  sure  how  often  they 
would  require  direct  assistance  from  the  NETWORK.  Approximately,  one  in 
five  educators  anticipated  using  the  Inquiry  Response  and  Referral  Services 
roughly  six  times  throughout  a  calendar  year.  Generally,  those  educators 
who  utilized  these  services  would  use  them  again  and  would  recommend  their 
use  to  others,  but  no  predictable  usage  patterns  could  be  noted.  Thus,  the 
following  recommendation  is  offered; 


Recommendation  7?  Conduct  a  follow-up  study  to  identify  why  the 
NETw6RK's  clients  have  difficulty  in  determining  how  often  they 
will  utilize  the  NETWORK'S  services  and  to  obtain  a  more 
accurate  estimate  of  the  projected  usage  rates  of  the  Information 
serv Ices. 

B.  Publication  Development  and  Dissemination  Service 

Military  educators  who  reviewed  and  judged  the  NETWORK'S  publications  were 
very  positive  about  their  quality  and  usefulness.  The  NETWORK  Fact  Sheet 
appeared  to  be  the  slightly  favored  publication.  All  publications  were 
rated  very  high  in  terms  of  quality  with  respect  to  comprehensiveness,  ease 
of  understanding,  accuracy  of  information,  and  format.  The  perceived 

t 

usefulness  of  the  NETWORK'S  publications  proved  to  be  above  average  in 
terms  of  learning  about  topics,  identifying  resources  and  applications  to 
educators'  work  responsibilities. 


Military  educators  indicated  that  the  NETWORK'S  publications  provided  them 
with  a  mechanism  through  which  they  were  able  to  identify  resources,  stay 
updated  on  topics,  and  follow  current  research  efforts.  In  addition,  the 
publications  were  seen  as  a  means  of  identifying  points-of-contact.  A 
large  majority  of  the  NETWORK'S  clients  shared  the  NETWORK'S  publications 
with  their  colleagues.  In  consideration  of  these  findings,  the  following 
recommendations  are  offered; 


Recommendation  8;  Continue  to  publish  and  disseminate  the 
NETWORK  Circuit  and  NETWORK  Fact  Sheet,  maintaining  their  current 
format. 

Recommendation  S:  Consider  expanding  the  length  of  the 
NETWORK  Circuit  to  include  additional  articles  on  educational 
resources  and  current  research  efforts. 

Recommendation  10;  Consider  publishing  the  NETWORK  Fact  Sheet  on 
a  monthly  basis  to  provide  educators  with  brief  updates  on 
current  issues  and  topics. 
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Current  Awareness  Service 


One  component  activity  of  the  NETWORK  service  included  the  development  and 
distribution  of  the  NETWORK  Vanguard.  Although  the  Vanguard  was  rated 
lower  in  comparison  to  the  other  NETWORK  publications,  military  educators 
still  rated  the  quality  and  usefulness  to  be  above  average.  Comments 
provided  by  some  educators  indicated  that  they  found  moderate  use  for 
information  about  current  journal  articles.  Given  these  positive,  yet 
mixed  reviews,  the  following  recommendations  are  offered; 


Recommendation  11:  Maintain  publication  of  the  NETWORK  Vanguard 


on  a  quarterly  basis. 

Recommendation  12;  In  consideration  of  an  expanded  version  of 
the  newsletter  and  monthly  releases  of  fact  sheets,  the  Vanguard 
could  be  terminated.  Selected  Information  cited  in  the  Vanguard 
could  be  incorporated  as  a  separate  column  in  the  newsletter. 


The  second  component  of  the  Current  Awareness  Service  involved  pilot 
testing  of  the  NETWORK  Profiles  Services.  About  one-half  of  the  small 
group  of  educators  provided  with  this  individualized  service  stated  they 
were  satisfied  with  the  information  materials  provided  to  them.  in 
addition,  one-half  of  the  participants  indicated  that  the  materials  were 
useful  to  their  professional  activities.  Comments  provided  by  some 
educators  were  very  enthusiastic  about  the  information  provided  to  them  and 
they  indicated  that  the  NETWORK  was  their  only  source  of  help. 


A  majority  of  the  participants  stated  that  they  shared  the  information 
materials  provided  to  them  through  the  Profiles  Service.  In  addition, 
two-thirds  of  the  participants  noted  that  the  Profile  Service  should  be 
continued  and  suggested  that  this  specialized  service  would  be  useful  to 
their  colleagues.  The  following  recommendation  is  offered: 


Recommendation  13t  The  NETWORK  Profiles  Service  should  not  be 
expanded  until  additional  pilot  testing  is  completed.  This 
specialized  service  should  be  tailored  to  meet  the  needs  of 
isolated  education  personnel,  thereby  avoiding  an  overlap  in 
information  services  available  to  more  accessible  personnel. 


uteri  zed  Database  Development 


A  primary  function  established  for  the  NETWORK  included  the  development  of 
a  computerized  database.  Information  citations  included  in  the  database 
were  designed  to  support  the  information  services  offered  by  the  NETWORK. 
Computer  costs  were  monitored  throughout  the  developmental  and  operational 
periods  associated  with  the  generation  of  the  database.  Overall,  computer 
usage  and  staff  charges  remained  within  projected  expenditures. 


Military  educators  were  asked  to  identify  information  categories  and 
subject  areas  that  were  of  most  interest  to  them.  Generally,  military 
educators  indicated  that  information  in  the  following  categories  would  be 
most  useful  to  them  in  the  operation  of  their  ACES  programs:  Education, 
Computer  Systems,  and  Management.  Within  the  Education  category,  primary 
areas  of  interest  included  information  about  Basic  Skills  Curricula,' 
Computer-Assisted  Instruction  and  Reading  Skills  Materials.  Secondary 
areas  of  interest  included  the  topics  of  Functional  Basic  Skills,  Quality 
Assurance  of  Programs,  Self-Paced  Instructional  Programs,  and  Tests  and 
Measurements.  Of  least  interest  to  educators  was  information  related  to 
topics  addressing  Psychomotor  Skills  and  Daily  Life  Coping  Skills. 


These  education  interests  appeared  to  be  fairly  stable  across  the  service 
branches.  Subject  areas  within  the  Computer  Systems  category  were 
indicated  to  be  high  areas  of  interest.  However,  a  wide  disparity  of 


interest  was  revealed  across  military  services.  Army,  Coast  Guard  and 
Marine  personnel  indicated  interest  in  all  related  subject  areas  such  as 
Computer  Equipment  Evaluations,  Computer  Systems  Capability,  Computer 
Systems  Evaluation,  and  Software  Evaluations.  Air  Force  and  Navy  personnel 
indicated  little,  if  any,  interest  in  these  topics. 


The  information  category  of  Management  was  rated  to  be  a  significant  area 
of  interest.  Generally  all  military  service  personnel  were  interested  in 
the  following  subject  areas:  Contracting  Requirements,  General  Management 
Skills,  Marketing  Educational  Programs,  Needs  Assessment  Techniques,  and 
Program  and  Curriculum  Evaluation.  Little  interest  was  noted  on  the  topics 
of  Economic  Analysis  and  Research  Methods. 


Finally,  two  other  areas  noted  to  be  of  interest  to  military  educators 
were;  Career  Planning  and  Guidance,  Information  on  Educational 
Institutions,  and  Military  Research  Programs.  In  consideration  of  these 
findings  the  following  recommendations  are  offered: 

Recommendation  1A;  Updates  to  and  expansions  of  the  NETWORK'S 
computerized  database  should  include  citations  of  information 
resources  that  address  the  following  topics: 

Education 

Basic  skills  curricula 
Computer-assisted  instruction 
Reading  skills 
Functional  basic  skills 
Quality  assurances  for  adult  programs 
Self-paced  instructional  programs 
Tests  and  measurements 

Computer  Systems 

Computer  equipment  evaluation 
Computer  systems  capability 
Computer  systems  evaluation 
Software  evaluations 
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Management 


Contracting  requirements 
General  management  skills 
Marketing  educational  programs 
Needs  assessment  techniques 
Program  and  curriculum  evaluation 

Counsel i ng 

Career  planning  and  guidance 
Civilian  labor  force  market  data 
Computer-based  guidance  systems 

Information  on  colleges,  vocational  schools,  and  other 
educational  institutions 

Government/Military  Information 

Military  research  and  programs 
State  educational  agencies  policies 


Recommendation  ISs  Format  the  NETWORK’S  publications  to  address 
the  categories  of  information  interests  stated  by  military 
educators.  Specifically,  the  NETWORK  Circuit  could  establish 
on-going  columns  such  as  the  "Computer  Corner"  and  a  "Management 
Corner."  Periodic  columns  that  address  secondary  topics  of 
interest  could  be  established  such  as  a  "Research  Corner"  and/or 
"Counseling  Corner." 


In  conclusion,  the  design  established  for  the  offering  of  information 
services  to  military  educators  proved  to  be  a  success.  Those  educators  who 
interacted  with  the  NETWORK  were  satisfied  with  the  services  provided  to 
them.  The  materials  and  publications  more  than  meet  the  needs  of  the 
network's  clients  and  have  been  judged  to  be  very  high  in  quality  and 
usefulness. 


One  lingering  and  basic  question  remains  relative  to  the  expected  volume  of 
information  requests  that  will  be  submitted  to  the  NETWORK  by  its  primary 
and  secondary  targeted  user  groups.  Data  gathered  throughout  the  initial 
operational  period  found  that  users  are  not  able  to  determine  how 
frequently  they  will  interact  with  the  NETWORK.  Additional  assessments 
need  to  be  undertaken  to  understand  the  difficulty  clients  have  in  judging 


111-10 


REFERENCES 


Russo,  R.P.,  Rivera,  C.,  DeCarme,  J.,  S  French,  A.  (in  press-a).  Basic 
skills  resource  center;  information  component  needs  assessment 
report .  Alexandria,  VA:  U.S.  Army  Research  Institute  For  the 
Behavioral  and  Social  Sciences. 


Rivera,  C.,  Russo,  R.P.,  &  OeCarme,  J.  (in  press-b).  Basic  ski  1  Is 

resource  center:  Military  educators  resource  NETWORK.  Alexandria, 
VA:  U.S.  Army  Research  institute  for  the  Behavioral  and  Social 
Sciences. 

Russo,  R.P.  (in  press-c).  Basic  skills  resource  center:  Military 

educators  resource  NETWORK  interim  report.  Rosslyn,  VA:  InterAmerica 
Research  Associates,  Inc. 


Russo,  R.P.,  Foster,  J.A.,  &  Modjeski,  R.B.  (In  press-d).  Basic  skills 
resource  center:  Documentation  and  phaseover  report  for  the  mi  liter 


educators  resource  NETWORK 


V-*.' 


KZLITASX  EDOOltOM  USOCICZ  NZIVOU  IllFOlllAnOII  UQUZST  FOIM 


Dtr«ctioaa .  pro^lda  tha  lafonaeioa  oiteliaad  balow  la  erdar  that  our  rMpenoa  co 
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Hew  aaay  elaaa  hava  you  uaad  our  sarvleaa?  first  eiaa  I  I  2«S  ciaaa  ^  aora 

II.  Is  cha  spaca  balow,  daseriba  your  raquasc  la  oarraclwa  fora. 


III.  Daseriba  cha  subjact  of  your  roduast  is  3  or  4  praclsa  caras  (a.c.,  raadlsf  skills, 
coapucar  aaaiscad  inscruccion,  adxile  licaraey,  ace.). 


17. 

Ara  you  iscarascad  in: 

7. 

Uhat  foras  of  Isforaacios  inter esc  you? 

□ 

Loeaciag  larga  duaaciclas  of 
.  rafaraneas  on  chis  cople? 

(How  aanv?  ) 

□ 

Cicacions  and  absersees  of  rasaareh  papers 
and  journal  artielas,  covering  c.ha  years 
froa  CO 

□ 

rinding  a  faw  of  cha  seat 
currant  rafaraneas? 

D 

□ 

Currant  research  project  suanarias. 

Sdueacion  prograa  deseripcions. 

□ 

□ 

rinding  a  faw  of  cha  "bast" 
icaas? 

Saing  added  co  sailing  list? 

□ 

Rafarral  co  ochar  sources  of  inforsacion 
(parsons  co' contact). 

71.  Plaaaa  daseriba  how  you  plan  co  usa  cha  laforaatlou  wa  provide,  and  provlda  us 
with  any  oc.har  laforaaclon  chat  nay  halp  us  uadarstand  your  raduasc. 
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□ 

Malliaf  List:  add  chg  ' 
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□ 
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f~1  Copy  from  MESM  ColltetioQ 


CCMMEMTS: 


n  S«areh  Rooulcs  (attach  copy) 
n  Othar  ____________ 


SEARCH  STHATSCT 


Evaluation  Form 


NETWORK  COMPUTER  SEARCH  EVALUATION 

Please  take  a  minute  to  answer  the  following  questions  about  the  results  of  the 
enclosed  online  search  completed  for  you.  Please  return  this  form  to  us  in  Che 
stamped,  self-addressed  return  envelope  attached  to  this  form.  Thank  you  for 
your  cooperation. 

1.  Were  you  satisfied  with  the  results  of  this  search?  YES  NO  □ 

2.  Were  the  results  of  this  search; 

TOO  NUMEROUS  □  SUFFICIENT  □  NOT  SUFFICIENT  □ 

3.  What  percentage  of  Che  ciCacions  were  relevant  to  your  needs?  _  5; 

k.  How  would  you  rate  the  response  time  to  this  request? 

EXCELLENT  Q  GOOD  Q  AVERAGE  □  POOR  □ 

5.  Did  you  receive  satisfactory  assistance  from  the  NETWORK  staff  when  you 
contacted  us  with  this  information  request? 

YES  □  NO  □ 

6.  Do  you  plan  to  use  the  NETWORK'S  services  again'’  YES  Q  NO  |  [ 

COMMENTS/SUGGESTIONS; 
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Telephone  Interview  Questions 


0  How  often  have  you  used  the  NETWORK’S  services? 

(Response  options:.  Once,  Twice,  More  than  twice) 

0  In  reference  to  the  materials  or  information  provided 
to  you  by  the  NETWORK  in  response  to  your  request,  how 
useful  were  the  materials?  (Response  options;  Extremely 
useful,  useful,  or  not  useful) 

0  How  would  you  rate  the  turnaround  time  or  response  time 
to  your  request? 

(Response  options:  Excellent,  good,  average,  or  poor) 

0  Did  the  information  the  NETWORK  provided  to  you... 
(Response  options:  Exceed  your  expectations,  meet 
your  expectations,  or  not  meet  your  expectations) 

0  How  helpful  were  the  NETWORK  staff  in  assisting  you  with 
your  information  needs? 

(Response  options:  Very  helpful,  helpful,  or  not  helpful) 

0  Do  you  plan  to  use  the  NETWORK  in  the  future? 

(Response  options:  Yes  or  no) 


Network 

User  Questionnaire 

The  Military  Educators  Resource  NETWORK  has  assisted  many  miii* 
tary  educators  in  identifying  information  resources  to  help  in  providing 
effective  educational  programs.  To  evaluate  the  effectiveness  of  our 
services,  we  would  appreciate  your  completing  this  questionnaire. 
Please  mark  an  "X”  in  the  appropriate  boxes  to  indicate  your 
responses. 

After  you  have  completed  this  form,  please  return  it  to  the  NETWORK 
at  the  address  indicated  below  using  a  government  franked  envelope. 
THANK  YOU! 

Military  Educators  Resource  NETWORK 
1555  Wilson  Boulevard.  Suite  508 
Rosslyn,  Virginia  22209 


A  Note  From 
HQDA  Education  Division 
(DAPE-MPE) 

Please  help  us  determine  the  effec¬ 
tiveness  of  the  NETWORK’S  serWces  by 
completing  and  returning  this  brief  form. 
Your  frank  responses  will  aid  in  the 
overall  evaluation  of  the  NETWORK. 
Your  prompt  anention  is  appreciated. 

BRUCE  T.  BATTEY  (LTC.  GS) 

Chief,  Education  Division 


1 .  What  is  your  title/position?  (check  one  only) 

Z  Education  Services  Officer 
Z  Education  Services  Specialist 
Z  Guidance  Counselor 


2.  What  is  your  military  service  affiliation?  (check  one  oniy) 

Z  Air  Force 
Z  Army 
Z  Coast  Guard 


Z  Researcher 
□  Administrator 
Z  Other  (specify) 


Z  Marines 
□  Navy 

Z  Other  (specify) 


3.  During  the  past  six  months,  how  many  questions  or  requests  have  you  submitted  to  the  NETWORK? 

Z  One  Request  Z  2-4  Requests  G  5  or  More  Requests  Z  Have  not  contacted  the  NETWORK 

(If  you  check  this  box.  skip  to  question  13) 

4.  On  the  average,  how  promptly  did  you  receive  a  response  to  your  request? 


_  Within  one  week  of  request 
Z  Within  two  weeks  of  request 


Z  Within  three  weeks  of  request 
Z  More  than  three  weeks  after  request 


5.  Indicate  your  level  of  satisfaction  with  the  speed  of  response. 


Extremely 

Good 


Acceptable 


Cannot  judge/ 
Not  applicable 


6.  How  did  you  use  the  information  you  obtained  from  the  NETWORK?  (check  ail  that  appiy) 

Z  Teaching  or  Training  Z  Administrative  Planning/Review 

Z  Research  Z  Personai 

Z  Counseiingrr esting  Z  Other  (specify)  _ 
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7.  Indicate  your  level  of  satififaction  with  the  information  you  received  in  response  to  your  request(s). 


Extremely 

Good  Acceptable 

5  4  3  2 

□  □  □  □ 


Very  Cannot  judge/ 

Poor  Not  applicable 

1  0 

□  □ 


8.  Have  you  shared  the  information  you  received  in  response  to  your  request(s)  with  others? 

pC  Yes  □  No 

I— »How  many  others? _ 


Did  you  find  the  format  and  organization  of 

□  Yes  »  —  Comments; 

□  No- 


10.  Were  the  points^f<contact  or  referrals  to  other  sources  of  information  provided  by  the  NETWORK  appropriate? 
□  Yes  □  No  C  Not  requested/not  applicable 


11.  Do  you  plan  to  use  the  NETWORK'S  services  again? 


a  Yes 

If  "Yes",  approximately  how  often? 

□  Once  a  week 

□  Once  a  month 

Six  or  more  times  during  a  calendar  year 
Z  Less  than  six  times  during  a  calendar  year 
C  Other  (specify) _ 


No 


Z  Undecided 

No”,  indicate  the  reason  why. 

□  Oo  not  need  it  any  longer 

C  Too  difficult  to  use  the  system 
C  Dissatisfied  with  previous  experience 

□  Other  (specify) _ 


12.  Would  you  recommend  the  NETWORK'S  services  to  others? 

Z  Yes  Z  No  C  Undecided 


13.  Please  provide  additional  comments  below. 


APPENDIX  F 


NETWORK  Reaction  Sheet 


Network 
Reaction  Sheet 

The  Military  Educators  Resource  NETWORK  develops  and 
distributes  a  variety  of  publications.  Those  include:  the  NETWORK 
Circuit,  a  quarterly  newsletter;  the  NETWORK  Fact  Sheet,  a  brief 
summary  of  a  high  interest  topic;  and  the  NETWORK  Vanguard, 
photocopies  of  current  professional  journal  tables^f-contents.  To 
evaluate  the  usefulness  of  these  publications,  we  would  appreciate 
your  completing  this  questionnaire.  Please  mark  an  “X"  in  the 
appropriate  boxes  to  indicate  your  response. 

After  you  have  completed  this  form,  please  return  it  to  the  NET¬ 
WORK  at  the  address  indicated  below  using  a  government  franked 
envelope.  THANK  YOU! 

Military  Educators  Resource  NETWORK 
1555  Wilson  Boulevard,  Suite  508 
Rosslyn,  Virginia  22209 

1 .  What  is  your  titte/position?  (check  one  only) 

□  Education  Services  Officer  □  Researcher 

□  Education  Services  Specialist  □  Administrator 

□  Guidance  Counselor  □  Other  (specify) 


2.  What  is  your  military  service  affiliation?  (check  one  only) 

□  Air  Force  □  Marines 

□  Army  □  Navy 

□  Coast  Guard  □  Other  (specify) 


A  Note  From 
HQOA  Education  Division 
(OAPE-MPE) 

PiMM  help  us  determine  the  useful¬ 
ness  of  the  NETWORK'S  publications 
by  completing  and  returning  this  brief 
form.  Your  frank  responses  wrill  aid  in 
the  overall  evaluation  of  the  NETWORK. 
Your  prompt  attention  is  appreciated. 

BRUCE  T.  BATTEY  (LTC,  GS) 

Chief,  Education  Division 


3.  For  each  of  the  NETWORK  publications,  please  indicate  the  level  of  helpfulness  each  provides. 


Very 

Moderately 

Not 

No  opinion/ 

Help^l 

Helpful 

Helpful 

Not  received 

5 

4 

3 

2 

1 

0 

NETWORK  Circuit 

□ 

□ 

□ 

f 

— 

NETWORK  Fact  Sheet 

rn 

C 

G 

□ 

C 

— 

NETWORK  Vanguard 

lJ 

□ 

— 

— 

4.  Considering  all  of  the  NETWORK’S  publications  as  a  whole,  please  indicate  your  level  of  satisfaction  with 


their  quality  and  usefulness. 

Extremely 

Good 

Acceptable 

Very 

Poor 

No  opinion 

5 

4 

3 

2 

1 

0 

QUALITY 

• 

a.  Comprehensiveness 

□ 

r? 

C 

G 

— 

b.  Ease  of  understanding 

C 

□ 

C 

G 

c.  Accuracy  of  information 

□ 

c 

□ 

G 

G 

d.  Format 

□ 

c 

□ 

□ 

G 

— 

Extramaiy 

Good 


Acceptablo 


Vory 

Poor  No  opinion 


USEFULNESS 

e.  Usafulnass  as  a  means  of 
learning  about  a  topic 

f.  Value  in  identifying 
resources  for  yourself 
and  others 

g.  Overall  usefulness  in 
your  work 


□  □  □ 


□  □  □ 


□  □  □ 


S.  How  do  you  use  the  NETWORK'S  publications?  (check  all  that  apply) 

□  Use  as  update  on  topics 

□  Use  to  identify  resources 

□  Use  to  follow  current  research  efforts 

□  Use  to  identify  points^f-contact 

□  Other  (specify)  _ - 


6.  Oo  you  share  the  NETWORK  publications  with  your  colleagues? 

1—0  Yes  □  No 


r-u  Tes 

I— •If  “Yes”,  on  an  average,  how  many 
colleagues  see  each  publication? 

□  1-2  0  3-4  0  5-6 


□  7-10 


□  10  + 


Please  provide  any  additional  comments  below. 


NETWORK  PROFILES  SERVICE  —  OPINION  FORM 

Tbe  Military  Educators  Resource  NETWORK  has  undertaken  a  pitot  test  of  a 
specialized  service  entitled  the  "NETWORK  Profiles."  Tou  were  selected  to 
participate  in  the  pilot  test  because  you  are  the  primary  contact  for  the 
education  programs  offered  at  your  installation.  The  Profiles  service  provides 
Army  educators  with  current  information  related  to  the  individual  educator's 
interests.  As  a  participant,  you  receive  information  periodically  about  current 
research  efforts,  publications,  and/or  other  resources  which  match  your  pre* 
stated  prof i le.  , 

Please  take  a  few  minutes  to  answer  the  following  questions  about  the  usefulness 
of  the  NETWORK  Profiles  service.  Place  an  "X"  in  the  appropriate  boxes  to 
indicate  your  responses.  Please  return  the  completed  form  to  the  NETWORK 
in  the  enclosed  stamped,  self-addressed  envelope.  THANK  YOU! 

1.  Were  you  satisfied  with  the  information  materials  provided  to  you? 

Very  Satisfied  Satisfied  Not  Satisfied  No  Opinion 

□  □  □  □ 

2.  How  useful  were  these  materials  to  your  professional  activities? 

Very  Useful  Useful  Not  Useful  No  Opinion 

□  □  D  □ 

3.  Did  you  share  these  materials  with  your  colleagues? 

Cn  Yes  Q  No 

^  If  "Yes",  how  many  colleagues  did 
you  share  the  information  with? 


□  1-2  Qj-'*  05-6  Q?-? 


□  10+ 


How  useful  would  the  NETWORK  Profiles  service  be  to  your  colleagues? 
(Please  provide  any  additional  comments  on  the  reverse  side  of  this  for-. 

Very  Useful  Useful  Not  Useful  No  Opinion 

□  □  □  □ 

Would  you  recommend  that  the  NETWORK  continue  to  offer  this  oecialized 
service? 

QYes  Q  No  Q  Undecided 

Please  provide  any  additional  comments  on  the  reverse  side  of  this  form. 


PT5585 


1555  VVilson  Boulevard 
Suite  508 

Rosslyn.  Virninia  2220P 

*o»  5:;oco: 


Subject  Area  Checklist 


To  further  develop  the  NETWORK  computerized  database,  we  need  to  know  the  topics  and/or  sub¬ 
ject  areas  that  are  of  primary  interest  to  you.  From  the  list  below,  indicate  those  areas  for  which  in¬ 
formation  would  be  beneflcial  to  you  in  the  operation  of  your  ACES  programs.  (Check  ail  that  apply.) 


Educatkw 

Q  Audiovisual  courseware  evaluations 
Q  Audiovisual  equipment  evaluations 
Q  Basic  skills  curricula,  instructional  materials  and  tests 
Q  Computer-assisted  instruction 

□  Curriculum  development 
Q  Curriculum  evaluations 

Q  Educational  achievement  levels 
Q  Educational  research 

Q  Functional  basic  skills,  including  job-related  training, 
occupational  planning,  functional  literacy  and  evaluation 
techniques 

Q  Individualized  instruction 

□  Instruction  design 

□  Literacy  standards 

Q  Psychology  of  learning 

□  (^ality  assuraiKe  for  adult  programs 
Q  Self-paced  instructional  programs 

Specific  skills: 

Q  Computation 

□  ESL 

Q  Listening 

□  Reading 
Q  Writing 

Q  Psychomotor 

□  Daily  life  coping 
Q  Teacher  evaluation 

□  Teaching  methods 

Q  Tests  and  measurements 
Q  Tuition  rates 

Q  OTHER  (specify); _ 


Computer  Systems 

Q  Computer  equipment  evaluations 
□  Computer  systems  compatibility 
O  Computer  systems  evaluation 
Q  Software  evaluations 
Q  OTHER  (specify):  _ 


Counseling 

Q  Career  maturity 
Q  Career  planning  and  guidance 
Q  Civilian  labor  force  market  dau 
Q  Computer-based  guidance  systems 
Q  Counseling  methods 
Q  Cross-cultural  counseling 

Q  Information  of  colleges,  vocational  schools  and  other 
educational  institutions 

□  OTHER  (specify):  _ 


Management 

S  Contracting  requirements 

Economic  analysis:  cost-benefit  analysis:  effectiveness 
analysis 

Q  General  management  skills 
Q  .Marketing  educational  programs 
Q  Needs  assessment  techniques 
Q  Program  and  curriculum  evaluation  techniques 
Q  Research  methods 

Q  OTHER  (specify);  _ 


Govemmcnt/Militaty  Information 

Q  Demographic  data 
Q  Directives  and  regulatory  information 
Q  Federal  budget  appropriations 
Q  Research  and  programs  in  other  Army  installations 
Q  Research  and  programs  in  other  branches  of  the  military 
Q  State  educational  agencies  policies:  educational  require¬ 
ments:  legal  issues 

Q  OTHER  (specify):  _ 


Title/Position; _ _ _ _ 

Military  Service  (circle  one):  Air  Force  Army  Coast  Guard  .Mannes  Navy 

Military  Installation 


Thanks  for  your  help! 

Once  completed,  place  this  checklist  in  a  government  franked  envelope  and  mail  using  the  enclosed 
label. 


